Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kvm-unit-tests/20250624.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-rt/20240806.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bullseye/20240129.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-h26forge/20250117.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'http://storage.kernelci.org/images/rootfs/debian/bookworm/20240313.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-fault-injection/20250520.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromiumos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'cros://chromeos-{krev}/{crosarch}/chromeos-{flavour}.flavour.config': 'krev' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-kselftest/20250724.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-ltp/20250818.0/{debarch}': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-gst-fluster/20240926.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' Format string error in param 'https://storage.kernelci.org/images/rootfs/debian/bookworm-wifi/20250925.0/{debarch}/': 'debarch' INFO: 172.21.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38418 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:23:01,285 - lava_callback - INFO - Received callback for node 68d8a9bc7aff7facd9a8d77a 2025-09-28 03:23:01,292 - lava_callback - INFO - Starting async job submit for node 68d8a9bc7aff7facd9a8d77a INFO: 172.21.0.1:45374 - "POST /node/68d8a9bc7aff7facd9a8d77a HTTP/1.0" 202 Accepted 2025-09-28 03:23:01,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:01,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9bc7aff7facd9a8d77a HTTP/1.1" 200 None 2025-09-28 03:23:01,326 - lava_callback - DEBUG - Processing node 68d8a9bc7aff7facd9a8d77a: baseline-arm64 2025-09-28 03:23:01,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:23:01,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:23:01,777 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a9bc7aff7facd9a8d77a/log.txt.gz 2025-09-28 03:23:01,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:23:02,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:23:02,044 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a9bc7aff7facd9a8d77a/lava_callback.json.gz 2025-09-28 03:23:02,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:02,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9bc7aff7facd9a8d77a HTTP/1.1" 200 None 2025-09-28 03:23:02,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:02,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:23:02,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:02,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a9bc7aff7facd9a8d77a HTTP/1.1" 200 None 2025-09-28 03:23:02,162 - lava_callback - INFO - Completed processing callback for node 68d8a9bc7aff7facd9a8d77a INFO: 172.21.0.1:45378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35700 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:23:29,589 - lava_callback - INFO - Received callback for node 68d8a9bb7aff7facd9a8d777 INFO: 172.21.0.1:33012 - "POST /node/68d8a9bb7aff7facd9a8d777 HTTP/1.0" 202 Accepted 2025-09-28 03:23:29,595 - lava_callback - INFO - Starting async job submit for node 68d8a9bb7aff7facd9a8d777 2025-09-28 03:23:29,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:29,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9bb7aff7facd9a8d777 HTTP/1.1" 200 None 2025-09-28 03:23:29,633 - lava_callback - DEBUG - Processing node 68d8a9bb7aff7facd9a8d777: baseline-arm64 2025-09-28 03:23:29,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:23:30,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:23:30,251 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a9bb7aff7facd9a8d777/log.txt.gz 2025-09-28 03:23:30,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:23:30,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:23:30,397 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a9bb7aff7facd9a8d777/lava_callback.json.gz 2025-09-28 03:23:30,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:30,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9bb7aff7facd9a8d777 HTTP/1.1" 200 None 2025-09-28 03:23:30,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:30,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:23:30,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:30,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a9bb7aff7facd9a8d777 HTTP/1.1" 200 None 2025-09-28 03:23:30,512 - lava_callback - INFO - Completed processing callback for node 68d8a9bb7aff7facd9a8d777 INFO: 172.21.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52924 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:23:48,167 - lava_callback - INFO - Received callback for node 68d7f9908dd6b39fd1aeb7d5 INFO: 172.21.0.1:52932 - "POST /node/68d7f9908dd6b39fd1aeb7d5 HTTP/1.0" 202 Accepted 2025-09-28 03:23:48,177 - lava_callback - INFO - Starting async job submit for node 68d7f9908dd6b39fd1aeb7d5 2025-09-28 03:23:48,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:48,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9908dd6b39fd1aeb7d5 HTTP/1.1" 200 None 2025-09-28 03:23:48,250 - lava_callback - DEBUG - Processing node 68d7f9908dd6b39fd1aeb7d5: baseline-arm64 2025-09-28 03:23:49,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:23:49,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:23:49,383 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9908dd6b39fd1aeb7d5/log.txt.gz 2025-09-28 03:23:49,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:23:49,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:23:49,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9908dd6b39fd1aeb7d5/lava_callback.json.gz DEBUG: Setup failed for 68d7f9908dd6b39fd1aeb7d5. Transitting job node result: incomplete -> fail 2025-09-28 03:23:49,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:49,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9908dd6b39fd1aeb7d5 HTTP/1.1" 200 None 2025-09-28 03:23:49,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:49,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 03:23:49,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:23:49,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7f9908dd6b39fd1aeb7d5 HTTP/1.1" 200 None 2025-09-28 03:23:49,788 - lava_callback - INFO - Completed processing callback for node 68d7f9908dd6b39fd1aeb7d5 INFO: 172.21.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46308 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:24:08,589 - lava_callback - INFO - Received callback for node 68d7f9908dd6b39fd1aeb7d8 INFO: 172.21.0.1:46324 - "POST /node/68d7f9908dd6b39fd1aeb7d8 HTTP/1.0" 202 Accepted 2025-09-28 03:24:08,598 - lava_callback - INFO - Starting async job submit for node 68d7f9908dd6b39fd1aeb7d8 2025-09-28 03:24:08,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:08,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9908dd6b39fd1aeb7d8 HTTP/1.1" 200 None 2025-09-28 03:24:08,656 - lava_callback - DEBUG - Processing node 68d7f9908dd6b39fd1aeb7d8: baseline-arm64 2025-09-28 03:24:09,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:09,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:09,696 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9908dd6b39fd1aeb7d8/log.txt.gz 2025-09-28 03:24:09,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:09,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:09,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9908dd6b39fd1aeb7d8/lava_callback.json.gz DEBUG: Setup failed for 68d7f9908dd6b39fd1aeb7d8. Transitting job node result: incomplete -> fail 2025-09-28 03:24:09,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:09,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9908dd6b39fd1aeb7d8 HTTP/1.1" 200 None 2025-09-28 03:24:09,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:09,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 03:24:09,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:10,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7f9908dd6b39fd1aeb7d8 HTTP/1.1" 200 None 2025-09-28 03:24:10,019 - lava_callback - INFO - Completed processing callback for node 68d7f9908dd6b39fd1aeb7d8 INFO: 172.21.0.1:47814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:24:19,237 - lava_callback - INFO - Received callback for node 68d8a8907aff7facd9a8d660 INFO: 172.21.0.1:47846 - "POST /node/68d8a8907aff7facd9a8d660 HTTP/1.0" 202 Accepted 2025-09-28 03:24:19,244 - lava_callback - INFO - Starting async job submit for node 68d8a8907aff7facd9a8d660 2025-09-28 03:24:19,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:19,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8907aff7facd9a8d660 HTTP/1.1" 200 None 2025-09-28 03:24:19,291 - lava_callback - DEBUG - Processing node 68d8a8907aff7facd9a8d660: baseline-arm64-broonie 2025-09-28 03:24:20,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:20,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:20,116 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8a8907aff7facd9a8d660/log.txt.gz 2025-09-28 03:24:20,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:20,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:20,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8a8907aff7facd9a8d660/lava_callback.json.gz 2025-09-28 03:24:20,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:20,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8907aff7facd9a8d660 HTTP/1.1" 200 None 2025-09-28 03:24:20,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:20,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:24:20,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:20,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a8907aff7facd9a8d660 HTTP/1.1" 200 None 2025-09-28 03:24:20,345 - lava_callback - INFO - Completed processing callback for node 68d8a8907aff7facd9a8d660 INFO: 172.21.0.1:43812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43890 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:24:40,931 - lava_callback - INFO - Received callback for node 68d8a8867aff7facd9a8d61f INFO: 172.21.0.1:48626 - "POST /node/68d8a8867aff7facd9a8d61f HTTP/1.0" 202 Accepted 2025-09-28 03:24:40,938 - lava_callback - INFO - Starting async job submit for node 68d8a8867aff7facd9a8d61f 2025-09-28 03:24:40,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:40,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8867aff7facd9a8d61f HTTP/1.1" 200 None 2025-09-28 03:24:40,980 - lava_callback - DEBUG - Processing node 68d8a8867aff7facd9a8d61f: baseline-arm64 2025-09-28 03:24:41,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:41,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:41,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8867aff7facd9a8d61f/log.txt.gz 2025-09-28 03:24:41,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:41,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:41,969 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8867aff7facd9a8d61f/lava_callback.json.gz DEBUG: 68d8a8867aff7facd9a8d61f Transitting job node result: pass -> incomplete 2025-09-28 03:24:41,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:42,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8867aff7facd9a8d61f HTTP/1.1" 200 None 2025-09-28 03:24:42,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:42,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:24:42,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:42,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a8867aff7facd9a8d61f HTTP/1.1" 200 None 2025-09-28 03:24:42,088 - lava_callback - INFO - Completed processing callback for node 68d8a8867aff7facd9a8d61f INFO: 172.21.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43938 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:24:53,858 - lava_callback - INFO - Received callback for node 68d8a8887aff7facd9a8d631 INFO: 172.21.0.1:43948 - "POST /node/68d8a8887aff7facd9a8d631 HTTP/1.0" 202 Accepted 2025-09-28 03:24:53,864 - lava_callback - INFO - Starting async job submit for node 68d8a8887aff7facd9a8d631 2025-09-28 03:24:53,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:53,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8887aff7facd9a8d631 HTTP/1.1" 200 None 2025-09-28 03:24:53,901 - lava_callback - DEBUG - Processing node 68d8a8887aff7facd9a8d631: baseline-arm64 2025-09-28 03:24:54,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:54,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:54,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8887aff7facd9a8d631/log.txt.gz 2025-09-28 03:24:54,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:54,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:54,411 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8887aff7facd9a8d631/lava_callback.json.gz 2025-09-28 03:24:54,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8887aff7facd9a8d631 HTTP/1.1" 200 None 2025-09-28 03:24:54,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:54,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:24:54,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:54,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a8887aff7facd9a8d631 HTTP/1.1" 200 None 2025-09-28 03:24:54,490 - lava_callback - INFO - Completed processing callback for node 68d8a8887aff7facd9a8d631 INFO: 172.21.0.1:43954 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:24:58,755 - lava_callback - INFO - Received callback for node 68d8a3a07aff7facd9a8d456 INFO: 172.21.0.1:43966 - "POST /node/68d8a3a07aff7facd9a8d456 HTTP/1.0" 202 Accepted 2025-09-28 03:24:58,761 - lava_callback - INFO - Starting async job submit for node 68d8a3a07aff7facd9a8d456 2025-09-28 03:24:58,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:58,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3a07aff7facd9a8d456 HTTP/1.1" 200 None 2025-09-28 03:24:58,804 - lava_callback - DEBUG - Processing node 68d8a3a07aff7facd9a8d456: baseline-arm 2025-09-28 03:24:59,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:59,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:59,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8a3a07aff7facd9a8d456/log.txt.gz 2025-09-28 03:24:59,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:24:59,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:24:59,355 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8a3a07aff7facd9a8d456/lava_callback.json.gz DEBUG: Setup failed for 68d8a3a07aff7facd9a8d456. Transitting job node result: incomplete -> fail 2025-09-28 03:24:59,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:59,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3a07aff7facd9a8d456 HTTP/1.1" 200 None 2025-09-28 03:24:59,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:59,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:24:59,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:24:59,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3a07aff7facd9a8d456 HTTP/1.1" 200 None 2025-09-28 03:24:59,464 - lava_callback - INFO - Completed processing callback for node 68d8a3a07aff7facd9a8d456 INFO: 172.21.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55084 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:26:05,020 - lava_callback - INFO - Received callback for node 68d8aa7aa178a5bbc0ea5c7f INFO: 172.21.0.1:58836 - "POST /node/68d8aa7aa178a5bbc0ea5c7f HTTP/1.0" 202 Accepted 2025-09-28 03:26:05,026 - lava_callback - INFO - Starting async job submit for node 68d8aa7aa178a5bbc0ea5c7f 2025-09-28 03:26:05,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:05,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa7aa178a5bbc0ea5c7f HTTP/1.1" 200 None 2025-09-28 03:26:05,059 - lava_callback - DEBUG - Processing node 68d8aa7aa178a5bbc0ea5c7f: baseline-arm64 2025-09-28 03:26:05,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:26:05,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:26:05,673 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8aa7aa178a5bbc0ea5c7f/log.txt.gz 2025-09-28 03:26:05,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:26:05,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:26:05,810 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8aa7aa178a5bbc0ea5c7f/lava_callback.json.gz 2025-09-28 03:26:05,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:05,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa7aa178a5bbc0ea5c7f HTTP/1.1" 200 None 2025-09-28 03:26:05,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:05,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:26:05,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:05,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8aa7aa178a5bbc0ea5c7f HTTP/1.1" 200 None 2025-09-28 03:26:05,941 - lava_callback - INFO - Completed processing callback for node 68d8aa7aa178a5bbc0ea5c7f INFO: 172.21.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52074 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:26:39,682 - lava_callback - INFO - Received callback for node 68d8aa65a178a5bbc0ea5c78 INFO: 172.21.0.1:60216 - "POST /node/68d8aa65a178a5bbc0ea5c78 HTTP/1.0" 202 Accepted 2025-09-28 03:26:39,688 - lava_callback - INFO - Starting async job submit for node 68d8aa65a178a5bbc0ea5c78 2025-09-28 03:26:39,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:39,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa65a178a5bbc0ea5c78 HTTP/1.1" 200 None 2025-09-28 03:26:39,723 - lava_callback - DEBUG - Processing node 68d8aa65a178a5bbc0ea5c78: baseline-arm64-broonie 2025-09-28 03:26:40,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:26:40,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:26:40,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8aa65a178a5bbc0ea5c78/log.txt.gz 2025-09-28 03:26:40,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:26:40,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:26:40,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8aa65a178a5bbc0ea5c78/lava_callback.json.gz 2025-09-28 03:26:40,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:40,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa65a178a5bbc0ea5c78 HTTP/1.1" 200 None 2025-09-28 03:26:40,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:40,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:26:40,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:40,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8aa65a178a5bbc0ea5c78 HTTP/1.1" 200 None 2025-09-28 03:26:40,454 - lava_callback - INFO - Completed processing callback for node 68d8aa65a178a5bbc0ea5c78 INFO: 172.21.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:26:59,856 - lava_callback - INFO - Received callback for node 68d8a9de7aff7facd9a8d780 INFO: 172.21.0.1:50352 - "POST /node/68d8a9de7aff7facd9a8d780 HTTP/1.0" 202 Accepted 2025-09-28 03:26:59,865 - lava_callback - INFO - Starting async job submit for node 68d8a9de7aff7facd9a8d780 2025-09-28 03:26:59,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:26:59,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9de7aff7facd9a8d780 HTTP/1.1" 200 None 2025-09-28 03:26:59,941 - lava_callback - DEBUG - Processing node 68d8a9de7aff7facd9a8d780: baseline-arm64-broonie 2025-09-28 03:27:00,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:27:00,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:27:00,848 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8a9de7aff7facd9a8d780/log.txt.gz 2025-09-28 03:27:00,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:27:00,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:27:00,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8a9de7aff7facd9a8d780/lava_callback.json.gz 2025-09-28 03:27:01,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:27:01,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9de7aff7facd9a8d780 HTTP/1.1" 200 None 2025-09-28 03:27:01,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:27:01,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:27:01,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:27:01,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a9de7aff7facd9a8d780 HTTP/1.1" 200 None 2025-09-28 03:27:01,071 - lava_callback - INFO - Completed processing callback for node 68d8a9de7aff7facd9a8d780 INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45512 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:27:59,306 - lava_callback - INFO - Received callback for node 68d8a3ab7aff7facd9a8d49b INFO: 172.21.0.1:42070 - "POST /node/68d8a3ab7aff7facd9a8d49b HTTP/1.0" 202 Accepted 2025-09-28 03:27:59,313 - lava_callback - INFO - Starting async job submit for node 68d8a3ab7aff7facd9a8d49b 2025-09-28 03:27:59,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:27:59,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ab7aff7facd9a8d49b HTTP/1.1" 200 None 2025-09-28 03:27:59,451 - lava_callback - DEBUG - Processing node 68d8a3ab7aff7facd9a8d49b: ltp-fs-bind 2025-09-28 03:27:59,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:28:00,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:28:00,114 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs-bind-68d8a3ab7aff7facd9a8d49b/log.txt.gz 2025-09-28 03:28:00,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:28:00,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:28:00,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs-bind-68d8a3ab7aff7facd9a8d49b/lava_callback.json.gz 2025-09-28 03:28:00,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:28:00,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ab7aff7facd9a8d49b HTTP/1.1" 200 None 2025-09-28 03:28:00,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:28:00,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:28:00,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:28:00,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ab7aff7facd9a8d49b HTTP/1.1" 200 None 2025-09-28 03:28:00,565 - lava_callback - INFO - Completed processing callback for node 68d8a3ab7aff7facd9a8d49b INFO: 172.21.0.1:42076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52120 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:28:56,705 - lava_callback - INFO - Received callback for node 68d8a6967aff7facd9a8d5d2 INFO: 172.21.0.1:52126 - "POST /node/68d8a6967aff7facd9a8d5d2 HTTP/1.0" 202 Accepted 2025-09-28 03:28:56,711 - lava_callback - INFO - Starting async job submit for node 68d8a6967aff7facd9a8d5d2 2025-09-28 03:28:56,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:28:56,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a6967aff7facd9a8d5d2 HTTP/1.1" 200 None 2025-09-28 03:28:56,760 - lava_callback - DEBUG - Processing node 68d8a6967aff7facd9a8d5d2: job-gcc-12-arm-612-cip 2025-09-28 03:28:57,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:28:57,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:28:57,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/job-gcc-12-arm-612-cip-68d8a6967aff7facd9a8d5d2/log.txt.gz 2025-09-28 03:28:57,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:28:57,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:28:57,311 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/job-gcc-12-arm-612-cip-68d8a6967aff7facd9a8d5d2/lava_callback.json.gz 2025-09-28 03:28:57,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:28:57,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a6967aff7facd9a8d5d2 HTTP/1.1" 200 None 2025-09-28 03:28:57,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:28:57,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f247aff7facd9a8d0e7 HTTP/1.1" 200 None 2025-09-28 03:28:57,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:28:57,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a6967aff7facd9a8d5d2 HTTP/1.1" 200 None 2025-09-28 03:28:57,442 - lava_callback - INFO - Completed processing callback for node 68d8a6967aff7facd9a8d5d2 INFO: 172.21.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:30:49,893 - lava_callback - INFO - Received callback for node 68d8a44f7aff7facd9a8d559 INFO: 172.21.0.1:33190 - "POST /node/68d8a44f7aff7facd9a8d559 HTTP/1.0" 202 Accepted 2025-09-28 03:30:49,940 - lava_callback - INFO - Starting async job submit for node 68d8a44f7aff7facd9a8d559 2025-09-28 03:30:49,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:30:49,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a44f7aff7facd9a8d559 HTTP/1.1" 200 None 2025-09-28 03:30:49,980 - lava_callback - DEBUG - Processing node 68d8a44f7aff7facd9a8d559: baseline-x86 2025-09-28 03:30:50,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:30:50,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:30:50,247 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8a44f7aff7facd9a8d559/log.txt.gz 2025-09-28 03:30:50,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:30:50,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:30:50,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8a44f7aff7facd9a8d559/lava_callback.json.gz 2025-09-28 03:30:50,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:30:50,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a44f7aff7facd9a8d559 HTTP/1.1" 200 None 2025-09-28 03:30:50,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:30:50,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f5b7aff7facd9a8d14f HTTP/1.1" 200 None 2025-09-28 03:30:50,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:30:50,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a44f7aff7facd9a8d559 HTTP/1.1" 200 None 2025-09-28 03:30:50,482 - lava_callback - INFO - Completed processing callback for node 68d8a44f7aff7facd9a8d559 INFO: 172.21.0.1:33196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46038 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:31:59,413 - lava_callback - INFO - Received callback for node 68d8ab05a178a5bbc0ea5cdb INFO: 172.21.0.1:53452 - "POST /node/68d8ab05a178a5bbc0ea5cdb HTTP/1.0" 202 Accepted 2025-09-28 03:31:59,420 - lava_callback - INFO - Starting async job submit for node 68d8ab05a178a5bbc0ea5cdb 2025-09-28 03:31:59,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:31:59,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ab05a178a5bbc0ea5cdb HTTP/1.1" 200 None 2025-09-28 03:31:59,466 - lava_callback - DEBUG - Processing node 68d8ab05a178a5bbc0ea5cdb: baseline-arm64-broonie 2025-09-28 03:31:59,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:32:00,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:32:00,032 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8ab05a178a5bbc0ea5cdb/log.txt.gz 2025-09-28 03:32:00,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:32:00,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:32:00,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8ab05a178a5bbc0ea5cdb/lava_callback.json.gz 2025-09-28 03:32:00,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:00,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ab05a178a5bbc0ea5cdb HTTP/1.1" 200 None 2025-09-28 03:32:00,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:00,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:32:00,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:00,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8ab05a178a5bbc0ea5cdb HTTP/1.1" 200 None 2025-09-28 03:32:00,301 - lava_callback - INFO - Completed processing callback for node 68d8ab05a178a5bbc0ea5cdb INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38324 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:32:23,946 - lava_callback - INFO - Received callback for node 68d8a3ac7aff7facd9a8d49e INFO: 172.21.0.1:33346 - "POST /node/68d8a3ac7aff7facd9a8d49e HTTP/1.0" 202 Accepted 2025-09-28 03:32:23,953 - lava_callback - INFO - Starting async job submit for node 68d8a3ac7aff7facd9a8d49e 2025-09-28 03:32:23,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:24,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ac7aff7facd9a8d49e HTTP/1.1" 200 None 2025-09-28 03:32:24,010 - lava_callback - DEBUG - Processing node 68d8a3ac7aff7facd9a8d49e: ltp-fsx 2025-09-28 03:32:24,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:32:24,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:32:24,466 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-68d8a3ac7aff7facd9a8d49e/log.txt.gz 2025-09-28 03:32:24,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:32:24,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:32:24,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-68d8a3ac7aff7facd9a8d49e/lava_callback.json.gz 2025-09-28 03:32:24,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:24,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ac7aff7facd9a8d49e HTTP/1.1" 200 None 2025-09-28 03:32:24,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:24,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:32:24,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:24,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ac7aff7facd9a8d49e HTTP/1.1" 200 None 2025-09-28 03:32:24,686 - lava_callback - INFO - Completed processing callback for node 68d8a3ac7aff7facd9a8d49e INFO: 172.21.0.1:33362 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:32:30,899 - lava_callback - INFO - Received callback for node 68d8a3aa7aff7facd9a8d492 INFO: 172.21.0.1:43828 - "POST /node/68d8a3aa7aff7facd9a8d492 HTTP/1.0" 202 Accepted 2025-09-28 03:32:30,913 - lava_callback - INFO - Starting async job submit for node 68d8a3aa7aff7facd9a8d492 2025-09-28 03:32:31,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:31,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3aa7aff7facd9a8d492 HTTP/1.1" 200 None 2025-09-28 03:32:31,260 - lava_callback - DEBUG - Processing node 68d8a3aa7aff7facd9a8d492: ltp-controllers 2025-09-28 03:32:34,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:32:34,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:32:34,518 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-68d8a3aa7aff7facd9a8d492/log.txt.gz 2025-09-28 03:32:34,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:32:34,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:32:34,749 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-68d8a3aa7aff7facd9a8d492/lava_callback.json.gz DEBUG: 68d8a3aa7aff7facd9a8d492 Transitting job node result: pass -> fail 2025-09-28 03:32:34,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:34,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3aa7aff7facd9a8d492 HTTP/1.1" 200 None 2025-09-28 03:32:34,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:34,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:32:34,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:32:35,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3aa7aff7facd9a8d492 HTTP/1.1" 200 None 2025-09-28 03:32:35,639 - lava_callback - INFO - Completed processing callback for node 68d8a3aa7aff7facd9a8d492 INFO: 172.21.0.1:43832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36726 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:33:17,059 - lava_callback - INFO - Received callback for node 68d8a8137aff7facd9a8d609 INFO: 172.21.0.1:36736 - "POST /node/68d8a8137aff7facd9a8d609 HTTP/1.0" 202 Accepted 2025-09-28 03:33:17,065 - lava_callback - INFO - Starting async job submit for node 68d8a8137aff7facd9a8d609 2025-09-28 03:33:17,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:33:17,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8137aff7facd9a8d609 HTTP/1.1" 200 None 2025-09-28 03:33:17,108 - lava_callback - DEBUG - Processing node 68d8a8137aff7facd9a8d609: baseline-arm 2025-09-28 03:33:17,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:33:17,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:33:17,677 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8a8137aff7facd9a8d609/log.txt.gz 2025-09-28 03:33:17,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:33:17,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:33:17,932 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8a8137aff7facd9a8d609/lava_callback.json.gz DEBUG: Setup failed for 68d8a8137aff7facd9a8d609. Transitting job node result: incomplete -> fail 2025-09-28 03:33:17,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:33:18,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8137aff7facd9a8d609 HTTP/1.1" 200 None 2025-09-28 03:33:18,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:33:18,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:33:18,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:33:18,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a8137aff7facd9a8d609 HTTP/1.1" 200 None 2025-09-28 03:33:18,110 - lava_callback - INFO - Completed processing callback for node 68d8a8137aff7facd9a8d609 INFO: 172.21.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33708 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:36:54,552 - lava_callback - INFO - Received callback for node 68d8a9327aff7facd9a8d756 INFO: 172.21.0.1:33712 - "POST /node/68d8a9327aff7facd9a8d756 HTTP/1.0" 202 Accepted 2025-09-28 03:36:54,559 - lava_callback - INFO - Starting async job submit for node 68d8a9327aff7facd9a8d756 2025-09-28 03:36:54,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:36:54,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9327aff7facd9a8d756 HTTP/1.1" 200 None 2025-09-28 03:36:54,614 - lava_callback - DEBUG - Processing node 68d8a9327aff7facd9a8d756: baseline-x86 2025-09-28 03:36:55,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:36:55,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:36:55,408 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8a9327aff7facd9a8d756/log.txt.gz 2025-09-28 03:36:55,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:36:55,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:36:55,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8a9327aff7facd9a8d756/lava_callback.json.gz DEBUG: Setup failed for 68d8a9327aff7facd9a8d756. Transitting job node result: incomplete -> fail 2025-09-28 03:36:55,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:36:55,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a9327aff7facd9a8d756 HTTP/1.1" 200 None 2025-09-28 03:36:55,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:36:55,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f5b7aff7facd9a8d14f HTTP/1.1" 200 None 2025-09-28 03:36:55,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:36:55,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a9327aff7facd9a8d756 HTTP/1.1" 200 None 2025-09-28 03:36:55,762 - lava_callback - INFO - Completed processing callback for node 68d8a9327aff7facd9a8d756 INFO: 172.21.0.1:33726 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:36:59,720 - lava_callback - INFO - Received callback for node 68d8a3ac7aff7facd9a8d4a1 INFO: 172.21.0.1:37648 - "POST /node/68d8a3ac7aff7facd9a8d4a1 HTTP/1.0" 202 Accepted 2025-09-28 03:36:59,727 - lava_callback - INFO - Starting async job submit for node 68d8a3ac7aff7facd9a8d4a1 2025-09-28 03:36:59,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:36:59,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ac7aff7facd9a8d4a1 HTTP/1.1" 200 None 2025-09-28 03:36:59,770 - lava_callback - DEBUG - Processing node 68d8a3ac7aff7facd9a8d4a1: ltp-input 2025-09-28 03:37:00,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:37:00,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:37:00,311 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-68d8a3ac7aff7facd9a8d4a1/log.txt.gz 2025-09-28 03:37:00,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:37:00,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:37:00,440 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-68d8a3ac7aff7facd9a8d4a1/lava_callback.json.gz 2025-09-28 03:37:00,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:00,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ac7aff7facd9a8d4a1 HTTP/1.1" 200 None 2025-09-28 03:37:00,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:00,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:37:00,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:00,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ac7aff7facd9a8d4a1 HTTP/1.1" 200 None 2025-09-28 03:37:00,555 - lava_callback - INFO - Completed processing callback for node 68d8a3ac7aff7facd9a8d4a1 INFO: 172.21.0.1:37652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37668 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:37:10,367 - lava_callback - INFO - Received callback for node 68d8a3ad7aff7facd9a8d4a4 INFO: 172.21.0.1:38722 - "POST /node/68d8a3ad7aff7facd9a8d4a4 HTTP/1.0" 202 Accepted 2025-09-28 03:37:10,374 - lava_callback - INFO - Starting async job submit for node 68d8a3ad7aff7facd9a8d4a4 2025-09-28 03:37:10,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:10,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ad7aff7facd9a8d4a4 HTTP/1.1" 200 None 2025-09-28 03:37:10,416 - lava_callback - DEBUG - Processing node 68d8a3ad7aff7facd9a8d4a4: ltp-ipc 2025-09-28 03:37:10,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:37:10,874 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:37:10,874 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-68d8a3ad7aff7facd9a8d4a4/log.txt.gz 2025-09-28 03:37:10,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:37:11,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:37:11,044 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-68d8a3ad7aff7facd9a8d4a4/lava_callback.json.gz 2025-09-28 03:37:11,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:11,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ad7aff7facd9a8d4a4 HTTP/1.1" 200 None 2025-09-28 03:37:11,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:11,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:37:11,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:11,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ad7aff7facd9a8d4a4 HTTP/1.1" 200 None 2025-09-28 03:37:11,148 - lava_callback - INFO - Completed processing callback for node 68d8a3ad7aff7facd9a8d4a4 INFO: 172.21.0.1:38726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:37:22,013 - lava_callback - INFO - Received callback for node 68d8ac30a178a5bbc0ea5d58 INFO: 172.21.0.1:47888 - "POST /node/68d8ac30a178a5bbc0ea5d58 HTTP/1.0" 202 Accepted 2025-09-28 03:37:22,019 - lava_callback - INFO - Starting async job submit for node 68d8ac30a178a5bbc0ea5d58 2025-09-28 03:37:22,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:22,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ac30a178a5bbc0ea5d58 HTTP/1.1" 200 None 2025-09-28 03:37:22,066 - lava_callback - DEBUG - Processing node 68d8ac30a178a5bbc0ea5d58: baseline-arm64-broonie 2025-09-28 03:37:22,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:37:22,622 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:37:22,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8ac30a178a5bbc0ea5d58/log.txt.gz 2025-09-28 03:37:22,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:37:22,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:37:22,761 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8ac30a178a5bbc0ea5d58/lava_callback.json.gz 2025-09-28 03:37:22,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:22,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ac30a178a5bbc0ea5d58 HTTP/1.1" 200 None 2025-09-28 03:37:22,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:22,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:37:22,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:37:22,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8ac30a178a5bbc0ea5d58 HTTP/1.1" 200 None 2025-09-28 03:37:22,859 - lava_callback - INFO - Completed processing callback for node 68d8ac30a178a5bbc0ea5d58 INFO: 172.21.0.1:47890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:39:24,775 - lava_callback - INFO - Received callback for node 68d8aa7ba178a5bbc0ea5c82 INFO: 172.21.0.1:54626 - "POST /node/68d8aa7ba178a5bbc0ea5c82 HTTP/1.0" 202 Accepted 2025-09-28 03:39:24,782 - lava_callback - INFO - Starting async job submit for node 68d8aa7ba178a5bbc0ea5c82 2025-09-28 03:39:24,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:39:24,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa7ba178a5bbc0ea5c82 HTTP/1.1" 200 None 2025-09-28 03:39:24,836 - lava_callback - DEBUG - Processing node 68d8aa7ba178a5bbc0ea5c82: baseline-arm64 2025-09-28 03:39:25,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:39:25,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:39:25,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8aa7ba178a5bbc0ea5c82/log.txt.gz 2025-09-28 03:39:25,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:39:26,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:39:26,099 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8aa7ba178a5bbc0ea5c82/lava_callback.json.gz DEBUG: 68d8aa7ba178a5bbc0ea5c82 Transitting job node result: pass -> incomplete 2025-09-28 03:39:26,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:39:26,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa7ba178a5bbc0ea5c82 HTTP/1.1" 200 None 2025-09-28 03:39:26,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:39:26,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:39:26,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:39:26,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8aa7ba178a5bbc0ea5c82 HTTP/1.1" 200 None 2025-09-28 03:39:26,216 - lava_callback - INFO - Completed processing callback for node 68d8aa7ba178a5bbc0ea5c82 INFO: 172.21.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:40:20,361 - lava_callback - INFO - Received callback for node 68d7f99a8dd6b39fd1aeb7de INFO: 172.21.0.1:58724 - "POST /node/68d7f99a8dd6b39fd1aeb7de HTTP/1.0" 202 Accepted 2025-09-28 03:40:20,368 - lava_callback - INFO - Starting async job submit for node 68d7f99a8dd6b39fd1aeb7de 2025-09-28 03:40:20,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:20,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f99a8dd6b39fd1aeb7de HTTP/1.1" 200 None 2025-09-28 03:40:20,421 - lava_callback - DEBUG - Processing node 68d7f99a8dd6b39fd1aeb7de: baseline-arm64 2025-09-28 03:40:21,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:40:21,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:40:21,461 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f99a8dd6b39fd1aeb7de/log.txt.gz 2025-09-28 03:40:21,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:40:21,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:40:21,645 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f99a8dd6b39fd1aeb7de/lava_callback.json.gz DEBUG: Setup failed for 68d7f99a8dd6b39fd1aeb7de. Transitting job node result: incomplete -> fail 2025-09-28 03:40:21,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:21,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f99a8dd6b39fd1aeb7de HTTP/1.1" 200 None 2025-09-28 03:40:21,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:21,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 03:40:21,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:21,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7f99a8dd6b39fd1aeb7de HTTP/1.1" 200 None 2025-09-28 03:40:21,780 - lava_callback - INFO - Completed processing callback for node 68d7f99a8dd6b39fd1aeb7de INFO: 172.21.0.1:58734 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:40:32,850 - lava_callback - INFO - Received callback for node 68d8adefa178a5bbc0ea6239 INFO: 172.21.0.1:47722 - "POST /node/68d8adefa178a5bbc0ea6239 HTTP/1.0" 202 Accepted 2025-09-28 03:40:32,856 - lava_callback - INFO - Starting async job submit for node 68d8adefa178a5bbc0ea6239 2025-09-28 03:40:32,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:32,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adefa178a5bbc0ea6239 HTTP/1.1" 200 None 2025-09-28 03:40:32,908 - lava_callback - DEBUG - Processing node 68d8adefa178a5bbc0ea6239: baseline-arm64 2025-09-28 03:40:33,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:40:33,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:40:33,427 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8adefa178a5bbc0ea6239/log.txt.gz 2025-09-28 03:40:33,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:40:33,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:40:33,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8adefa178a5bbc0ea6239/lava_callback.json.gz 2025-09-28 03:40:33,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:33,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adefa178a5bbc0ea6239 HTTP/1.1" 200 None 2025-09-28 03:40:33,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:33,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:40:33,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:33,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8adefa178a5bbc0ea6239 HTTP/1.1" 200 None 2025-09-28 03:40:33,693 - lava_callback - INFO - Completed processing callback for node 68d8adefa178a5bbc0ea6239 INFO: 172.21.0.1:47732 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:40:43,273 - lava_callback - INFO - Received callback for node 68d8adeea178a5bbc0ea6236 INFO: 172.21.0.1:34454 - "POST /node/68d8adeea178a5bbc0ea6236 HTTP/1.0" 202 Accepted 2025-09-28 03:40:43,279 - lava_callback - INFO - Starting async job submit for node 68d8adeea178a5bbc0ea6236 2025-09-28 03:40:43,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:43,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adeea178a5bbc0ea6236 HTTP/1.1" 200 None 2025-09-28 03:40:43,313 - lava_callback - DEBUG - Processing node 68d8adeea178a5bbc0ea6236: baseline-arm64 2025-09-28 03:40:43,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:40:43,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:40:43,781 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8adeea178a5bbc0ea6236/log.txt.gz 2025-09-28 03:40:43,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:40:43,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:40:43,938 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8adeea178a5bbc0ea6236/lava_callback.json.gz 2025-09-28 03:40:43,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:43,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adeea178a5bbc0ea6236 HTTP/1.1" 200 None 2025-09-28 03:40:43,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:43,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 03:40:43,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:40:44,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8adeea178a5bbc0ea6236 HTTP/1.1" 200 None 2025-09-28 03:40:44,013 - lava_callback - INFO - Completed processing callback for node 68d8adeea178a5bbc0ea6236 INFO: 172.21.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:41:19,989 - lava_callback - INFO - Received callback for node 68d8adeaa178a5bbc0ea622c INFO: 172.21.0.1:55332 - "POST /node/68d8adeaa178a5bbc0ea622c HTTP/1.0" 202 Accepted 2025-09-28 03:41:19,994 - lava_callback - INFO - Starting async job submit for node 68d8adeaa178a5bbc0ea622c 2025-09-28 03:41:20,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:20,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adeaa178a5bbc0ea622c HTTP/1.1" 200 None 2025-09-28 03:41:20,030 - lava_callback - DEBUG - Processing node 68d8adeaa178a5bbc0ea622c: baseline-arm 2025-09-28 03:41:20,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:41:20,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:41:20,344 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8adeaa178a5bbc0ea622c/log.txt.gz 2025-09-28 03:41:20,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:41:20,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:41:20,461 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8adeaa178a5bbc0ea622c/lava_callback.json.gz 2025-09-28 03:41:20,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:20,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adeaa178a5bbc0ea622c HTTP/1.1" 200 None 2025-09-28 03:41:20,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:20,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae6a178a5bbc0ea5cb4 HTTP/1.1" 200 None 2025-09-28 03:41:20,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:20,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8adeaa178a5bbc0ea622c HTTP/1.1" 200 None 2025-09-28 03:41:20,537 - lava_callback - INFO - Completed processing callback for node 68d8adeaa178a5bbc0ea622c INFO: 172.21.0.1:55338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46430 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:41:50,802 - lava_callback - INFO - Received callback for node 68d8a3ab7aff7facd9a8d495 INFO: 172.21.0.1:57212 - "POST /node/68d8a3ab7aff7facd9a8d495 HTTP/1.0" 202 Accepted 2025-09-28 03:41:50,809 - lava_callback - INFO - Starting async job submit for node 68d8a3ab7aff7facd9a8d495 2025-09-28 03:41:50,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:50,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ab7aff7facd9a8d495 HTTP/1.1" 200 None 2025-09-28 03:41:50,947 - lava_callback - DEBUG - Processing node 68d8a3ab7aff7facd9a8d495: ltp-cve 2025-09-28 03:41:51,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:41:51,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:41:51,856 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-68d8a3ab7aff7facd9a8d495/log.txt.gz 2025-09-28 03:41:51,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:41:52,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:41:52,015 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-68d8a3ab7aff7facd9a8d495/lava_callback.json.gz DEBUG: 68d8a3ab7aff7facd9a8d495 Transitting job node result: pass -> fail 2025-09-28 03:41:52,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:52,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ab7aff7facd9a8d495 HTTP/1.1" 200 None 2025-09-28 03:41:52,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:52,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:41:52,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:41:52,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ab7aff7facd9a8d495 HTTP/1.1" 200 None 2025-09-28 03:41:52,259 - lava_callback - INFO - Completed processing callback for node 68d8a3ab7aff7facd9a8d495 INFO: 172.21.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54126 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:44:29,430 - lava_callback - INFO - Received callback for node 68d7f99a8dd6b39fd1aeb7e1 INFO: 172.21.0.1:48804 - "POST /node/68d7f99a8dd6b39fd1aeb7e1 HTTP/1.0" 202 Accepted 2025-09-28 03:44:29,437 - lava_callback - INFO - Starting async job submit for node 68d7f99a8dd6b39fd1aeb7e1 2025-09-28 03:44:29,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:29,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f99a8dd6b39fd1aeb7e1 HTTP/1.1" 200 None 2025-09-28 03:44:29,515 - lava_callback - DEBUG - Processing node 68d7f99a8dd6b39fd1aeb7e1: baseline-arm64 2025-09-28 03:44:30,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:44:30,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:44:30,514 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f99a8dd6b39fd1aeb7e1/log.txt.gz 2025-09-28 03:44:30,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:44:30,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:44:30,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f99a8dd6b39fd1aeb7e1/lava_callback.json.gz DEBUG: Setup failed for 68d7f99a8dd6b39fd1aeb7e1. Transitting job node result: incomplete -> fail 2025-09-28 03:44:30,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:30,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f99a8dd6b39fd1aeb7e1 HTTP/1.1" 200 None 2025-09-28 03:44:30,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:30,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 03:44:30,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:30,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7f99a8dd6b39fd1aeb7e1 HTTP/1.1" 200 None 2025-09-28 03:44:30,855 - lava_callback - INFO - Completed processing callback for node 68d7f99a8dd6b39fd1aeb7e1 INFO: 172.21.0.1:48818 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:44:39,241 - lava_callback - INFO - Received callback for node 68d8a3a27aff7facd9a8d462 INFO: 172.21.0.1:48822 - "POST /node/68d8a3a27aff7facd9a8d462 HTTP/1.0" 202 Accepted 2025-09-28 03:44:39,247 - lava_callback - INFO - Starting async job submit for node 68d8a3a27aff7facd9a8d462 2025-09-28 03:44:39,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:39,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3a27aff7facd9a8d462 HTTP/1.1" 200 None 2025-09-28 03:44:39,299 - lava_callback - DEBUG - Processing node 68d8a3a27aff7facd9a8d462: baseline-arm 2025-09-28 03:44:39,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:44:39,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:44:39,696 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8a3a27aff7facd9a8d462/log.txt.gz 2025-09-28 03:44:39,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:44:39,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:44:39,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8a3a27aff7facd9a8d462/lava_callback.json.gz DEBUG: Setup failed for 68d8a3a27aff7facd9a8d462. Transitting job node result: incomplete -> fail 2025-09-28 03:44:39,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:39,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3a27aff7facd9a8d462 HTTP/1.1" 200 None 2025-09-28 03:44:39,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:39,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:44:39,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:39,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3a27aff7facd9a8d462 HTTP/1.1" 200 None 2025-09-28 03:44:39,939 - lava_callback - INFO - Completed processing callback for node 68d8a3a27aff7facd9a8d462 INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:44:49,691 - lava_callback - INFO - Received callback for node 68d7f9c28dd6b39fd1aeb7e7 INFO: 172.21.0.1:34380 - "POST /node/68d7f9c28dd6b39fd1aeb7e7 HTTP/1.0" 202 Accepted 2025-09-28 03:44:49,698 - lava_callback - INFO - Starting async job submit for node 68d7f9c28dd6b39fd1aeb7e7 2025-09-28 03:44:49,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:49,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9c28dd6b39fd1aeb7e7 HTTP/1.1" 200 None 2025-09-28 03:44:49,750 - lava_callback - DEBUG - Processing node 68d7f9c28dd6b39fd1aeb7e7: baseline-arm64 2025-09-28 03:44:50,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:44:50,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:44:50,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9c28dd6b39fd1aeb7e7/log.txt.gz 2025-09-28 03:44:50,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:44:50,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:44:50,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9c28dd6b39fd1aeb7e7/lava_callback.json.gz DEBUG: Setup failed for 68d7f9c28dd6b39fd1aeb7e7. Transitting job node result: incomplete -> fail 2025-09-28 03:44:51,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:51,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9c28dd6b39fd1aeb7e7 HTTP/1.1" 200 None 2025-09-28 03:44:51,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:51,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 03:44:51,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:44:51,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7f9c28dd6b39fd1aeb7e7 HTTP/1.1" 200 None 2025-09-28 03:44:51,130 - lava_callback - INFO - Completed processing callback for node 68d7f9c28dd6b39fd1aeb7e7 INFO: 172.21.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53202 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:46:41,767 - lava_callback - INFO - Received callback for node 68d8a3ad7aff7facd9a8d4a7 INFO: 172.21.0.1:39184 - "POST /node/68d8a3ad7aff7facd9a8d4a7 HTTP/1.0" 202 Accepted 2025-09-28 03:46:41,774 - lava_callback - INFO - Starting async job submit for node 68d8a3ad7aff7facd9a8d4a7 2025-09-28 03:46:41,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:46:41,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ad7aff7facd9a8d4a7 HTTP/1.1" 200 None 2025-09-28 03:46:41,837 - lava_callback - DEBUG - Processing node 68d8a3ad7aff7facd9a8d4a7: ltp-pty 2025-09-28 03:46:42,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:46:42,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:46:42,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-68d8a3ad7aff7facd9a8d4a7/log.txt.gz 2025-09-28 03:46:42,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:46:42,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:46:42,549 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-68d8a3ad7aff7facd9a8d4a7/lava_callback.json.gz DEBUG: 68d8a3ad7aff7facd9a8d4a7 Transitting job node result: pass -> fail 2025-09-28 03:46:42,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:46:42,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ad7aff7facd9a8d4a7 HTTP/1.1" 200 None 2025-09-28 03:46:42,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:46:42,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:46:42,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:46:42,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ad7aff7facd9a8d4a7 HTTP/1.1" 200 None 2025-09-28 03:46:42,727 - lava_callback - INFO - Completed processing callback for node 68d8a3ad7aff7facd9a8d4a7 INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37534 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:50:10,903 - lava_callback - INFO - Received callback for node 68d8a3ae7aff7facd9a8d4ad INFO: 172.21.0.1:60970 - "POST /node/68d8a3ae7aff7facd9a8d4ad HTTP/1.0" 202 Accepted 2025-09-28 03:50:10,909 - lava_callback - INFO - Starting async job submit for node 68d8a3ae7aff7facd9a8d4ad 2025-09-28 03:50:10,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:50:10,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ae7aff7facd9a8d4ad HTTP/1.1" 200 None 2025-09-28 03:50:10,956 - lava_callback - DEBUG - Processing node 68d8a3ae7aff7facd9a8d4ad: ltp-smoketest 2025-09-28 03:50:11,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:50:11,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:50:11,531 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8a3ae7aff7facd9a8d4ad/log.txt.gz 2025-09-28 03:50:11,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:50:11,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:50:11,677 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8a3ae7aff7facd9a8d4ad/lava_callback.json.gz DEBUG: 68d8a3ae7aff7facd9a8d4ad Transitting job node result: pass -> fail 2025-09-28 03:50:11,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:50:11,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ae7aff7facd9a8d4ad HTTP/1.1" 200 None 2025-09-28 03:50:11,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:50:11,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:50:11,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:50:11,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ae7aff7facd9a8d4ad HTTP/1.1" 200 None 2025-09-28 03:50:11,810 - lava_callback - INFO - Completed processing callback for node 68d8a3ae7aff7facd9a8d4ad INFO: 172.21.0.1:60978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60182 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:51:55,883 - lava_callback - INFO - Received callback for node 68d8ac7ea178a5bbc0ea6206 INFO: 172.21.0.1:39098 - "POST /node/68d8ac7ea178a5bbc0ea6206 HTTP/1.0" 202 Accepted 2025-09-28 03:51:55,890 - lava_callback - INFO - Starting async job submit for node 68d8ac7ea178a5bbc0ea6206 2025-09-28 03:51:55,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:51:55,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ac7ea178a5bbc0ea6206 HTTP/1.1" 200 None 2025-09-28 03:51:55,943 - lava_callback - DEBUG - Processing node 68d8ac7ea178a5bbc0ea6206: baseline-arm 2025-09-28 03:51:56,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:51:56,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:51:56,447 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8ac7ea178a5bbc0ea6206/log.txt.gz 2025-09-28 03:51:56,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:51:56,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:51:56,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8ac7ea178a5bbc0ea6206/lava_callback.json.gz DEBUG: Setup failed for 68d8ac7ea178a5bbc0ea6206. Transitting job node result: incomplete -> fail 2025-09-28 03:51:56,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:51:56,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ac7ea178a5bbc0ea6206 HTTP/1.1" 200 None 2025-09-28 03:51:56,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:51:56,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:51:56,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:51:56,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8ac7ea178a5bbc0ea6206 HTTP/1.1" 200 None 2025-09-28 03:51:56,738 - lava_callback - INFO - Completed processing callback for node 68d8ac7ea178a5bbc0ea6206 INFO: 172.21.0.1:39110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45190 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:52:34,455 - lava_callback - INFO - Received callback for node 68d8ab79a178a5bbc0ea5d47 INFO: 172.21.0.1:51670 - "POST /node/68d8ab79a178a5bbc0ea5d47 HTTP/1.0" 202 Accepted 2025-09-28 03:52:34,467 - lava_callback - INFO - Starting async job submit for node 68d8ab79a178a5bbc0ea5d47 2025-09-28 03:52:34,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:52:34,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ab79a178a5bbc0ea5d47 HTTP/1.1" 200 None 2025-09-28 03:52:34,589 - lava_callback - DEBUG - Processing node 68d8ab79a178a5bbc0ea5d47: job-gcc-12-arm-612-cip 2025-09-28 03:52:35,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:52:35,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:52:35,400 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/job-gcc-12-arm-612-cip-68d8ab79a178a5bbc0ea5d47/log.txt.gz 2025-09-28 03:52:35,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:52:35,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:52:35,707 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/job-gcc-12-arm-612-cip-68d8ab79a178a5bbc0ea5d47/lava_callback.json.gz 2025-09-28 03:52:35,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:52:35,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ab79a178a5bbc0ea5d47 HTTP/1.1" 200 None 2025-09-28 03:52:35,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:52:35,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f247aff7facd9a8d0e7 HTTP/1.1" 200 None 2025-09-28 03:52:35,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:52:35,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8ab79a178a5bbc0ea5d47 HTTP/1.1" 200 None 2025-09-28 03:52:35,997 - lava_callback - INFO - Completed processing callback for node 68d8ab79a178a5bbc0ea5d47 INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47430 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:54:25,780 - lava_callback - INFO - Received callback for node 68d8a6ea7aff7facd9a8d5e7 INFO: 172.21.0.1:45200 - "POST /node/68d8a6ea7aff7facd9a8d5e7 HTTP/1.0" 202 Accepted 2025-09-28 03:54:25,831 - lava_callback - INFO - Starting async job submit for node 68d8a6ea7aff7facd9a8d5e7 2025-09-28 03:54:25,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:25,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a6ea7aff7facd9a8d5e7 HTTP/1.1" 200 None 2025-09-28 03:54:25,898 - lava_callback - DEBUG - Processing node 68d8a6ea7aff7facd9a8d5e7: job-gcc-12-x86-612-cip 2025-09-28 03:54:26,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45202 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:54:26,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:54:26,916 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/job-gcc-12-x86-612-cip-68d8a6ea7aff7facd9a8d5e7/log.txt.gz 2025-09-28 03:54:26,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:54:27,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:54:27,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/job-gcc-12-x86-612-cip-68d8a6ea7aff7facd9a8d5e7/lava_callback.json.gz 2025-09-28 03:54:27,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:27,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a6ea7aff7facd9a8d5e7 HTTP/1.1" 200 None 2025-09-28 03:54:27,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:27,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f277aff7facd9a8d0f0 HTTP/1.1" 200 None 2025-09-28 03:54:27,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:27,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a6ea7aff7facd9a8d5e7 HTTP/1.1" 200 None 2025-09-28 03:54:27,622 - lava_callback - INFO - Completed processing callback for node 68d8a6ea7aff7facd9a8d5e7 2025-09-28 03:54:31,095 - lava_callback - INFO - Received callback for node 68d8a3b07aff7facd9a8d4b6 INFO: 172.21.0.1:40070 - "POST /node/68d8a3b07aff7facd9a8d4b6 HTTP/1.0" 202 Accepted 2025-09-28 03:54:31,106 - lava_callback - INFO - Starting async job submit for node 68d8a3b07aff7facd9a8d4b6 2025-09-28 03:54:31,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:31,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3b07aff7facd9a8d4b6 HTTP/1.1" 200 None 2025-09-28 03:54:31,180 - lava_callback - DEBUG - Processing node 68d8a3b07aff7facd9a8d4b6: ltp-timers 2025-09-28 03:54:31,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:54:32,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:54:32,292 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-68d8a3b07aff7facd9a8d4b6/log.txt.gz 2025-09-28 03:54:32,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:54:32,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:54:32,985 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-68d8a3b07aff7facd9a8d4b6/lava_callback.json.gz 2025-09-28 03:54:33,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:33,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3b07aff7facd9a8d4b6 HTTP/1.1" 200 None 2025-09-28 03:54:33,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:33,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:54:33,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:54:33,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3b07aff7facd9a8d4b6 HTTP/1.1" 200 None 2025-09-28 03:54:33,104 - lava_callback - INFO - Completed processing callback for node 68d8a3b07aff7facd9a8d4b6 INFO: 172.21.0.1:40072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60330 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:55:18,115 - lava_callback - INFO - Received callback for node 68d8b188a178a5bbc0ea6326 INFO: 172.21.0.1:60332 - "POST /node/68d8b188a178a5bbc0ea6326 HTTP/1.0" 202 Accepted 2025-09-28 03:55:18,123 - lava_callback - INFO - Starting async job submit for node 68d8b188a178a5bbc0ea6326 2025-09-28 03:55:18,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:55:18,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b188a178a5bbc0ea6326 HTTP/1.1" 200 None 2025-09-28 03:55:18,165 - lava_callback - DEBUG - Processing node 68d8b188a178a5bbc0ea6326: baseline-x86 2025-09-28 03:55:18,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:55:18,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:55:18,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b188a178a5bbc0ea6326/log.txt.gz 2025-09-28 03:55:18,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:55:18,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:55:18,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b188a178a5bbc0ea6326/lava_callback.json.gz 2025-09-28 03:55:18,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:55:18,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b188a178a5bbc0ea6326 HTTP/1.1" 200 None 2025-09-28 03:55:18,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:55:18,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aafca178a5bbc0ea5cd2 HTTP/1.1" 200 None 2025-09-28 03:55:18,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:55:18,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b188a178a5bbc0ea6326 HTTP/1.1" 200 None 2025-09-28 03:55:18,723 - lava_callback - INFO - Completed processing callback for node 68d8b188a178a5bbc0ea6326 INFO: 172.21.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:56:31,175 - lava_callback - INFO - Received callback for node 68d8a3ae7aff7facd9a8d4aa INFO: 172.21.0.1:45560 - "POST /node/68d8a3ae7aff7facd9a8d4aa HTTP/1.0" 202 Accepted 2025-09-28 03:56:31,185 - lava_callback - INFO - Starting async job submit for node 68d8a3ae7aff7facd9a8d4aa 2025-09-28 03:56:31,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:31,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ae7aff7facd9a8d4aa HTTP/1.1" 200 None 2025-09-28 03:56:31,264 - lava_callback - DEBUG - Processing node 68d8a3ae7aff7facd9a8d4aa: ltp-sched 2025-09-28 03:56:31,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:31,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:31,980 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-68d8a3ae7aff7facd9a8d4aa/log.txt.gz 2025-09-28 03:56:31,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:32,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:32,167 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-68d8a3ae7aff7facd9a8d4aa/lava_callback.json.gz 2025-09-28 03:56:32,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:32,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3ae7aff7facd9a8d4aa HTTP/1.1" 200 None 2025-09-28 03:56:32,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:32,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:56:32,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:32,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3ae7aff7facd9a8d4aa HTTP/1.1" 200 None 2025-09-28 03:56:32,335 - lava_callback - INFO - Completed processing callback for node 68d8a3ae7aff7facd9a8d4aa INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:56:42,493 - lava_callback - INFO - Received callback for node 68d8b1bca178a5bbc0ea634a INFO: 172.21.0.1:52404 - "POST /node/68d8b1bca178a5bbc0ea634a HTTP/1.0" 202 Accepted 2025-09-28 03:56:42,504 - lava_callback - INFO - Starting async job submit for node 68d8b1bca178a5bbc0ea634a 2025-09-28 03:56:42,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:42,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bca178a5bbc0ea634a HTTP/1.1" 200 None 2025-09-28 03:56:42,576 - lava_callback - DEBUG - Processing node 68d8b1bca178a5bbc0ea634a: baseline-arm 2025-09-28 03:56:43,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:43,083 - lava_callback - INFO - Received callback for node 68d8b1cfa178a5bbc0ea63c8 2025-09-28 03:56:43,135 - lava_callback - INFO - Starting async job submit for node 68d8b1cfa178a5bbc0ea63c8 INFO: 172.21.0.1:52412 - "POST /node/68d8b1cfa178a5bbc0ea63c8 HTTP/1.0" 202 Accepted 2025-09-28 03:56:43,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:43,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cfa178a5bbc0ea63c8 HTTP/1.1" 200 None 2025-09-28 03:56:43,194 - lava_callback - DEBUG - Processing node 68d8b1cfa178a5bbc0ea63c8: baseline-arm-pengutronix 2025-09-28 03:56:43,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:43,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bca178a5bbc0ea634a/log.txt.gz 2025-09-28 03:56:43,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:43,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:43,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:43,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bca178a5bbc0ea634a/lava_callback.json.gz 2025-09-28 03:56:43,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:43,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:43,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b1cfa178a5bbc0ea63c8/log.txt.gz 2025-09-28 03:56:43,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bca178a5bbc0ea634a HTTP/1.1" 200 None 2025-09-28 03:56:43,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:43,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:43,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:56:43,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:43,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bca178a5bbc0ea634a HTTP/1.1" 200 None 2025-09-28 03:56:43,857 - lava_callback - INFO - Completed processing callback for node 68d8b1bca178a5bbc0ea634a 2025-09-28 03:56:43,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:43,908 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b1cfa178a5bbc0ea63c8/lava_callback.json.gz 2025-09-28 03:56:43,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:43,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cfa178a5bbc0ea63c8 HTTP/1.1" 200 None 2025-09-28 03:56:43,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:44,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:56:44,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:44,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1cfa178a5bbc0ea63c8 HTTP/1.1" 200 None 2025-09-28 03:56:44,062 - lava_callback - INFO - Completed processing callback for node 68d8b1cfa178a5bbc0ea63c8 INFO: 172.21.0.1:52414 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:56:48,550 - lava_callback - INFO - Received callback for node 68d8b1bea178a5bbc0ea6356 INFO: 172.21.0.1:52428 - "POST /node/68d8b1bea178a5bbc0ea6356 HTTP/1.0" 202 Accepted 2025-09-28 03:56:48,559 - lava_callback - INFO - Starting async job submit for node 68d8b1bea178a5bbc0ea6356 2025-09-28 03:56:48,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:48,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bea178a5bbc0ea6356 HTTP/1.1" 200 None 2025-09-28 03:56:48,619 - lava_callback - DEBUG - Processing node 68d8b1bea178a5bbc0ea6356: baseline-arm 2025-09-28 03:56:49,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:49,157 - lava_callback - INFO - Received callback for node 68d8b1cea178a5bbc0ea63c5 INFO: 172.21.0.1:52444 - "POST /node/68d8b1cea178a5bbc0ea63c5 HTTP/1.0" 202 Accepted 2025-09-28 03:56:49,168 - lava_callback - INFO - Starting async job submit for node 68d8b1cea178a5bbc0ea63c5 2025-09-28 03:56:49,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:49,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cea178a5bbc0ea63c5 HTTP/1.1" 200 None 2025-09-28 03:56:49,218 - lava_callback - DEBUG - Processing node 68d8b1cea178a5bbc0ea63c5: baseline-arm-pengutronix 2025-09-28 03:56:49,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:49,322 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bea178a5bbc0ea6356/log.txt.gz 2025-09-28 03:56:49,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:49,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:49,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:49,667 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bea178a5bbc0ea6356/lava_callback.json.gz 2025-09-28 03:56:49,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:49,696 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b1cea178a5bbc0ea63c5/log.txt.gz 2025-09-28 03:56:49,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:49,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:56:49,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bea178a5bbc0ea6356 HTTP/1.1" 200 None 2025-09-28 03:56:49,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:49,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:56:49,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:49,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bea178a5bbc0ea6356 HTTP/1.1" 200 None 2025-09-28 03:56:49,777 - lava_callback - INFO - Completed processing callback for node 68d8b1bea178a5bbc0ea6356 2025-09-28 03:56:50,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:56:50,032 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b1cea178a5bbc0ea63c5/lava_callback.json.gz 2025-09-28 03:56:50,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:50,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cea178a5bbc0ea63c5 HTTP/1.1" 200 None 2025-09-28 03:56:50,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:50,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:56:50,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:56:50,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1cea178a5bbc0ea63c5 HTTP/1.1" 200 None 2025-09-28 03:56:50,184 - lava_callback - INFO - Completed processing callback for node 68d8b1cea178a5bbc0ea63c5 INFO: 172.21.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40404 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:57:20,691 - lava_callback - INFO - Received callback for node 68d8b1c3a178a5bbc0ea637c INFO: 172.21.0.1:46172 - "POST /node/68d8b1c3a178a5bbc0ea637c HTTP/1.0" 202 Accepted 2025-09-28 03:57:20,700 - lava_callback - INFO - Starting async job submit for node 68d8b1c3a178a5bbc0ea637c 2025-09-28 03:57:20,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:20,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c3a178a5bbc0ea637c HTTP/1.1" 200 None 2025-09-28 03:57:20,763 - lava_callback - DEBUG - Processing node 68d8b1c3a178a5bbc0ea637c: baseline-arm-broonie 2025-09-28 03:57:21,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:21,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:21,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d8b1c3a178a5bbc0ea637c/log.txt.gz 2025-09-28 03:57:21,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:22,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:22,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d8b1c3a178a5bbc0ea637c/lava_callback.json.gz 2025-09-28 03:57:22,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:22,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c3a178a5bbc0ea637c HTTP/1.1" 200 None 2025-09-28 03:57:22,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:22,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:22,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:22,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1c3a178a5bbc0ea637c HTTP/1.1" 200 None 2025-09-28 03:57:22,397 - lava_callback - INFO - Completed processing callback for node 68d8b1c3a178a5bbc0ea637c INFO: 172.21.0.1:46174 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:57:32,267 - lava_callback - INFO - Received callback for node 68d8b1c8a178a5bbc0ea639b INFO: 172.21.0.1:41286 - "POST /node/68d8b1c8a178a5bbc0ea639b HTTP/1.0" 202 Accepted 2025-09-28 03:57:32,276 - lava_callback - INFO - Starting async job submit for node 68d8b1c8a178a5bbc0ea639b 2025-09-28 03:57:32,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:32,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c8a178a5bbc0ea639b HTTP/1.1" 200 None 2025-09-28 03:57:32,343 - lava_callback - DEBUG - Processing node 68d8b1c8a178a5bbc0ea639b: baseline-arm-clabbe 2025-09-28 03:57:32,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:33,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:33,478 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d8b1c8a178a5bbc0ea639b/log.txt.gz 2025-09-28 03:57:33,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:34,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:34,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d8b1c8a178a5bbc0ea639b/lava_callback.json.gz 2025-09-28 03:57:34,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:34,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c8a178a5bbc0ea639b HTTP/1.1" 200 None 2025-09-28 03:57:34,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:34,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:34,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:34,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1c8a178a5bbc0ea639b HTTP/1.1" 200 None 2025-09-28 03:57:34,291 - lava_callback - INFO - Completed processing callback for node 68d8b1c8a178a5bbc0ea639b INFO: 172.21.0.1:41300 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:57:38,065 - lava_callback - INFO - Received callback for node 68d8b1cca178a5bbc0ea63b9 INFO: 172.21.0.1:41304 - "POST /node/68d8b1cca178a5bbc0ea63b9 HTTP/1.0" 202 Accepted 2025-09-28 03:57:38,076 - lava_callback - INFO - Starting async job submit for node 68d8b1cca178a5bbc0ea63b9 2025-09-28 03:57:38,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:38,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cca178a5bbc0ea63b9 HTTP/1.1" 200 None 2025-09-28 03:57:38,146 - lava_callback - DEBUG - Processing node 68d8b1cca178a5bbc0ea63b9: baseline-arm-kontron 2025-09-28 03:57:38,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:39,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:39,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d8b1cca178a5bbc0ea63b9/log.txt.gz 2025-09-28 03:57:39,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:39,315 - lava_callback - INFO - Received callback for node 68d8b1cda178a5bbc0ea63bc INFO: 172.21.0.1:45088 - "POST /node/68d8b1cda178a5bbc0ea63bc HTTP/1.0" 202 Accepted 2025-09-28 03:57:39,324 - lava_callback - INFO - Starting async job submit for node 68d8b1cda178a5bbc0ea63bc 2025-09-28 03:57:39,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:39,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cda178a5bbc0ea63bc HTTP/1.1" 200 None 2025-09-28 03:57:39,388 - lava_callback - DEBUG - Processing node 68d8b1cda178a5bbc0ea63bc: baseline-arm-kontron 2025-09-28 03:57:39,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:39,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:39,911 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d8b1cca178a5bbc0ea63b9/lava_callback.json.gz 2025-09-28 03:57:39,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:39,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cca178a5bbc0ea63b9 HTTP/1.1" 200 None 2025-09-28 03:57:39,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:40,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:40,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:40,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1cca178a5bbc0ea63b9 HTTP/1.1" 200 None 2025-09-28 03:57:40,042 - lava_callback - INFO - Completed processing callback for node 68d8b1cca178a5bbc0ea63b9 2025-09-28 03:57:40,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:40,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d8b1cda178a5bbc0ea63bc/log.txt.gz 2025-09-28 03:57:40,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:40,588 - lava_callback - INFO - Received callback for node 68d8b1c4a178a5bbc0ea6380 INFO: 172.21.0.1:45100 - "POST /node/68d8b1c4a178a5bbc0ea6380 HTTP/1.0" 202 Accepted 2025-09-28 03:57:40,597 - lava_callback - INFO - Starting async job submit for node 68d8b1c4a178a5bbc0ea6380 2025-09-28 03:57:40,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:40,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c4a178a5bbc0ea6380 HTTP/1.1" 200 None 2025-09-28 03:57:40,649 - lava_callback - DEBUG - Processing node 68d8b1c4a178a5bbc0ea6380: baseline-arm-broonie 2025-09-28 03:57:41,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:41,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:41,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-68d8b1cda178a5bbc0ea63bc/lava_callback.json.gz 2025-09-28 03:57:41,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:41,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cda178a5bbc0ea63bc HTTP/1.1" 200 None 2025-09-28 03:57:41,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:41,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:41,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:41,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1cda178a5bbc0ea63bc HTTP/1.1" 200 None 2025-09-28 03:57:41,345 - lava_callback - INFO - Completed processing callback for node 68d8b1cda178a5bbc0ea63bc 2025-09-28 03:57:41,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:41,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d8b1c4a178a5bbc0ea6380/log.txt.gz 2025-09-28 03:57:41,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:42,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:42,212 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-68d8b1c4a178a5bbc0ea6380/lava_callback.json.gz 2025-09-28 03:57:42,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:42,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c4a178a5bbc0ea6380 HTTP/1.1" 200 None 2025-09-28 03:57:42,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:42,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:42,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:42,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1c4a178a5bbc0ea6380 HTTP/1.1" 200 None 2025-09-28 03:57:42,341 - lava_callback - INFO - Completed processing callback for node 68d8b1c4a178a5bbc0ea6380 2025-09-28 03:57:45,382 - lava_callback - INFO - Received callback for node 68d8b1bea178a5bbc0ea635c INFO: 172.21.0.1:45110 - "POST /node/68d8b1bea178a5bbc0ea635c HTTP/1.0" 202 Accepted 2025-09-28 03:57:45,391 - lava_callback - INFO - Starting async job submit for node 68d8b1bea178a5bbc0ea635c 2025-09-28 03:57:45,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:45,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bea178a5bbc0ea635c HTTP/1.1" 200 None 2025-09-28 03:57:45,457 - lava_callback - DEBUG - Processing node 68d8b1bea178a5bbc0ea635c: baseline-arm-baylibre 2025-09-28 03:57:45,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:46,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:46,409 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1bea178a5bbc0ea635c/log.txt.gz 2025-09-28 03:57:46,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45118 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:57:46,814 - lava_callback - INFO - Received callback for node 68d8b1bfa178a5bbc0ea635f INFO: 172.21.0.1:45120 - "POST /node/68d8b1bfa178a5bbc0ea635f HTTP/1.0" 202 Accepted 2025-09-28 03:57:46,825 - lava_callback - INFO - Starting async job submit for node 68d8b1bfa178a5bbc0ea635f 2025-09-28 03:57:46,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:46,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bfa178a5bbc0ea635f HTTP/1.1" 200 None 2025-09-28 03:57:46,901 - lava_callback - DEBUG - Processing node 68d8b1bfa178a5bbc0ea635f: baseline-arm-baylibre 2025-09-28 03:57:47,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:47,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:47,392 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1bea178a5bbc0ea635c/lava_callback.json.gz 2025-09-28 03:57:47,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:47,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bea178a5bbc0ea635c HTTP/1.1" 200 None 2025-09-28 03:57:47,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:47,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:47,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:47,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bea178a5bbc0ea635c HTTP/1.1" 200 None 2025-09-28 03:57:47,516 - lava_callback - INFO - Completed processing callback for node 68d8b1bea178a5bbc0ea635c 2025-09-28 03:57:48,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:48,096 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1bfa178a5bbc0ea635f/log.txt.gz 2025-09-28 03:57:48,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:48,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:48,635 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1bfa178a5bbc0ea635f/lava_callback.json.gz 2025-09-28 03:57:48,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:48,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bfa178a5bbc0ea635f HTTP/1.1" 200 None 2025-09-28 03:57:48,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:48,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:48,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:48,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bfa178a5bbc0ea635f HTTP/1.1" 200 None 2025-09-28 03:57:48,752 - lava_callback - INFO - Completed processing callback for node 68d8b1bfa178a5bbc0ea635f 2025-09-28 03:57:51,797 - lava_callback - INFO - Received callback for node 68d8b1c1a178a5bbc0ea636e INFO: 172.21.0.1:47918 - "POST /node/68d8b1c1a178a5bbc0ea636e HTTP/1.0" 202 Accepted 2025-09-28 03:57:51,808 - lava_callback - INFO - Starting async job submit for node 68d8b1c1a178a5bbc0ea636e 2025-09-28 03:57:51,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:51,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c1a178a5bbc0ea636e HTTP/1.1" 200 None 2025-09-28 03:57:51,882 - lava_callback - DEBUG - Processing node 68d8b1c1a178a5bbc0ea636e: baseline-arm-baylibre 2025-09-28 03:57:52,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:52,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:52,822 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c1a178a5bbc0ea636e/log.txt.gz 2025-09-28 03:57:52,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:57:53,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:57:53,318 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c1a178a5bbc0ea636e/lava_callback.json.gz 2025-09-28 03:57:53,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:53,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c1a178a5bbc0ea636e HTTP/1.1" 200 None 2025-09-28 03:57:53,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:53,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:57:53,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:53,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1c1a178a5bbc0ea636e HTTP/1.1" 200 None 2025-09-28 03:57:53,436 - lava_callback - INFO - Completed processing callback for node 68d8b1c1a178a5bbc0ea636e INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47940 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:57:59,335 - lava_callback - INFO - Received callback for node 68d8b1cda178a5bbc0ea63bf INFO: 172.21.0.1:54916 - "POST /node/68d8b1cda178a5bbc0ea63bf HTTP/1.0" 202 Accepted 2025-09-28 03:57:59,344 - lava_callback - INFO - Starting async job submit for node 68d8b1cda178a5bbc0ea63bf 2025-09-28 03:57:59,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:57:59,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cda178a5bbc0ea63bf HTTP/1.1" 200 None 2025-09-28 03:57:59,421 - lava_callback - DEBUG - Processing node 68d8b1cda178a5bbc0ea63bf: baseline-arm-pengutronix 2025-09-28 03:57:59,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:58:00,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:00,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b1cda178a5bbc0ea63bf/log.txt.gz 2025-09-28 03:58:00,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:58:00,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:00,926 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b1cda178a5bbc0ea63bf/lava_callback.json.gz 2025-09-28 03:58:01,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:01,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cda178a5bbc0ea63bf HTTP/1.1" 200 None 2025-09-28 03:58:01,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:01,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:58:01,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:01,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1cda178a5bbc0ea63bf HTTP/1.1" 200 None 2025-09-28 03:58:01,105 - lava_callback - INFO - Completed processing callback for node 68d8b1cda178a5bbc0ea63bf 2025-09-28 03:58:05,884 - lava_callback - INFO - Received callback for node 68d8b1bba178a5bbc0ea6344 INFO: 172.21.0.1:54930 - "POST /node/68d8b1bba178a5bbc0ea6344 HTTP/1.0" 202 Accepted 2025-09-28 03:58:05,894 - lava_callback - INFO - Starting async job submit for node 68d8b1bba178a5bbc0ea6344 2025-09-28 03:58:05,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:05,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bba178a5bbc0ea6344 HTTP/1.1" 200 None 2025-09-28 03:58:05,993 - lava_callback - DEBUG - Processing node 68d8b1bba178a5bbc0ea6344: baseline-arm 2025-09-28 03:58:06,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:54938 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:58:06,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:06,813 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bba178a5bbc0ea6344/log.txt.gz 2025-09-28 03:58:06,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:58:07,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:07,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bba178a5bbc0ea6344/lava_callback.json.gz 2025-09-28 03:58:07,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:07,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bba178a5bbc0ea6344 HTTP/1.1" 200 None 2025-09-28 03:58:07,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:07,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:58:07,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bba178a5bbc0ea6344 HTTP/1.1" 200 None 2025-09-28 03:58:07,200 - lava_callback - INFO - Completed processing callback for node 68d8b1bba178a5bbc0ea6344 INFO: 172.21.0.1:54946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38020 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:58:17,470 - lava_callback - INFO - Received callback for node 68d8b1c0a178a5bbc0ea6365 INFO: 172.21.0.1:38022 - "POST /node/68d8b1c0a178a5bbc0ea6365 HTTP/1.0" 202 Accepted 2025-09-28 03:58:17,479 - lava_callback - INFO - Starting async job submit for node 68d8b1c0a178a5bbc0ea6365 2025-09-28 03:58:17,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:17,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c0a178a5bbc0ea6365 HTTP/1.1" 200 None 2025-09-28 03:58:17,553 - lava_callback - DEBUG - Processing node 68d8b1c0a178a5bbc0ea6365: baseline-arm-baylibre 2025-09-28 03:58:17,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:58:18,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:18,418 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c0a178a5bbc0ea6365/log.txt.gz 2025-09-28 03:58:18,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:58:18,737 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:18,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c0a178a5bbc0ea6365/lava_callback.json.gz 2025-09-28 03:58:18,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:18,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c0a178a5bbc0ea6365 HTTP/1.1" 200 None 2025-09-28 03:58:18,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:18,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:58:18,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:18,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1c0a178a5bbc0ea6365 HTTP/1.1" 200 None 2025-09-28 03:58:18,872 - lava_callback - INFO - Completed processing callback for node 68d8b1c0a178a5bbc0ea6365 INFO: 172.21.0.1:49964 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:58:33,403 - lava_callback - INFO - Received callback for node 68d8b1c0a178a5bbc0ea6362 INFO: 172.21.0.1:60050 - "POST /node/68d8b1c0a178a5bbc0ea6362 HTTP/1.0" 202 Accepted 2025-09-28 03:58:33,409 - lava_callback - INFO - Starting async job submit for node 68d8b1c0a178a5bbc0ea6362 2025-09-28 03:58:33,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:33,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c0a178a5bbc0ea6362 HTTP/1.1" 200 None 2025-09-28 03:58:33,442 - lava_callback - DEBUG - Processing node 68d8b1c0a178a5bbc0ea6362: baseline-arm-baylibre 2025-09-28 03:58:33,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:58:33,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:33,757 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c0a178a5bbc0ea6362/log.txt.gz 2025-09-28 03:58:33,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:58:33,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:58:33,865 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c0a178a5bbc0ea6362/lava_callback.json.gz 2025-09-28 03:58:33,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:33,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c0a178a5bbc0ea6362 HTTP/1.1" 200 None 2025-09-28 03:58:33,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:33,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:58:33,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:58:33,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1c0a178a5bbc0ea6362 HTTP/1.1" 200 None 2025-09-28 03:58:33,942 - lava_callback - INFO - Completed processing callback for node 68d8b1c0a178a5bbc0ea6362 INFO: 172.21.0.1:60054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:59:01,146 - lava_callback - INFO - Received callback for node 68d8adeba178a5bbc0ea622f INFO: 172.21.0.1:60318 - "POST /node/68d8adeba178a5bbc0ea622f HTTP/1.0" 202 Accepted 2025-09-28 03:59:01,153 - lava_callback - INFO - Starting async job submit for node 68d8adeba178a5bbc0ea622f 2025-09-28 03:59:01,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:01,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adeba178a5bbc0ea622f HTTP/1.1" 200 None 2025-09-28 03:59:01,270 - lava_callback - DEBUG - Processing node 68d8adeba178a5bbc0ea622f: ltp-smoketest 2025-09-28 03:59:01,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:01,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:01,887 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8adeba178a5bbc0ea622f/log.txt.gz 2025-09-28 03:59:01,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:02,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:02,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8adeba178a5bbc0ea622f/lava_callback.json.gz 2025-09-28 03:59:02,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:02,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8adeba178a5bbc0ea622f HTTP/1.1" 200 None 2025-09-28 03:59:02,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:02,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae6a178a5bbc0ea5cb4 HTTP/1.1" 200 None 2025-09-28 03:59:02,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:02,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8adeba178a5bbc0ea622f HTTP/1.1" 200 None 2025-09-28 03:59:02,241 - lava_callback - INFO - Completed processing callback for node 68d8adeba178a5bbc0ea622f 2025-09-28 03:59:02,984 - lava_callback - INFO - Received callback for node 68d8b24aa178a5bbc0ea6439 INFO: 172.21.0.1:60334 - "POST /node/68d8b24aa178a5bbc0ea6439 HTTP/1.0" 202 Accepted 2025-09-28 03:59:02,990 - lava_callback - INFO - Starting async job submit for node 68d8b24aa178a5bbc0ea6439 2025-09-28 03:59:03,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:03,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b24aa178a5bbc0ea6439 HTTP/1.1" 200 None 2025-09-28 03:59:03,022 - lava_callback - DEBUG - Processing node 68d8b24aa178a5bbc0ea6439: baseline-arm-pengutronix 2025-09-28 03:59:03,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:03,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:03,334 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b24aa178a5bbc0ea6439/log.txt.gz 2025-09-28 03:59:03,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:03,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:03,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-68d8b24aa178a5bbc0ea6439/lava_callback.json.gz 2025-09-28 03:59:03,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:03,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b24aa178a5bbc0ea6439 HTTP/1.1" 200 None 2025-09-28 03:59:03,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:03,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:59:03,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:03,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b24aa178a5bbc0ea6439 HTTP/1.1" 200 None 2025-09-28 03:59:03,539 - lava_callback - INFO - Completed processing callback for node 68d8b24aa178a5bbc0ea6439 INFO: 172.21.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK 2025-09-28 03:59:11,243 - lava_callback - INFO - Received callback for node 68d8a3b07aff7facd9a8d4b9 INFO: 172.21.0.1:41538 - "POST /node/68d8a3b07aff7facd9a8d4b9 HTTP/1.0" 202 Accepted 2025-09-28 03:59:11,250 - lava_callback - INFO - Starting async job submit for node 68d8a3b07aff7facd9a8d4b9 2025-09-28 03:59:11,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:11,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3b07aff7facd9a8d4b9 HTTP/1.1" 200 None 2025-09-28 03:59:11,328 - lava_callback - DEBUG - Processing node 68d8a3b07aff7facd9a8d4b9: ltp-watchqueue 2025-09-28 03:59:11,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:11,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:11,827 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-68d8a3b07aff7facd9a8d4b9/log.txt.gz 2025-09-28 03:59:11,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:11,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:11,969 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-68d8a3b07aff7facd9a8d4b9/lava_callback.json.gz 2025-09-28 03:59:12,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:12,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3b07aff7facd9a8d4b9 HTTP/1.1" 200 None 2025-09-28 03:59:12,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:12,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 03:59:12,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:12,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3b07aff7facd9a8d4b9 HTTP/1.1" 200 None 2025-09-28 03:59:12,081 - lava_callback - INFO - Completed processing callback for node 68d8a3b07aff7facd9a8d4b9 INFO: 172.21.0.1:41540 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 03:59:15,363 - lava_callback - INFO - Received callback for node 68d8b1caa178a5bbc0ea63ad INFO: 172.21.0.1:41548 - "POST /node/68d8b1caa178a5bbc0ea63ad HTTP/1.0" 202 Accepted 2025-09-28 03:59:15,369 - lava_callback - INFO - Starting async job submit for node 68d8b1caa178a5bbc0ea63ad 2025-09-28 03:59:15,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:15,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1caa178a5bbc0ea63ad HTTP/1.1" 200 None 2025-09-28 03:59:15,403 - lava_callback - DEBUG - Processing node 68d8b1caa178a5bbc0ea63ad: baseline-arm-clabbe 2025-09-28 03:59:15,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:15,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:15,714 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d8b1caa178a5bbc0ea63ad/log.txt.gz 2025-09-28 03:59:15,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 03:59:15,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 03:59:15,826 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d8b1caa178a5bbc0ea63ad/lava_callback.json.gz 2025-09-28 03:59:15,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:15,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1caa178a5bbc0ea63ad HTTP/1.1" 200 None 2025-09-28 03:59:15,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:15,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 03:59:15,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 03:59:15,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1caa178a5bbc0ea63ad HTTP/1.1" 200 None 2025-09-28 03:59:15,935 - lava_callback - INFO - Completed processing callback for node 68d8b1caa178a5bbc0ea63ad INFO: 172.21.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34462 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:00:18,415 - lava_callback - INFO - Received callback for node 68d8b1bda178a5bbc0ea6350 INFO: 172.21.0.1:34468 - "POST /node/68d8b1bda178a5bbc0ea6350 HTTP/1.0" 202 Accepted 2025-09-28 04:00:18,421 - lava_callback - INFO - Starting async job submit for node 68d8b1bda178a5bbc0ea6350 2025-09-28 04:00:18,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:18,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bda178a5bbc0ea6350 HTTP/1.1" 200 None 2025-09-28 04:00:18,461 - lava_callback - DEBUG - Processing node 68d8b1bda178a5bbc0ea6350: baseline-arm 2025-09-28 04:00:18,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:00:18,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:00:18,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bda178a5bbc0ea6350/log.txt.gz 2025-09-28 04:00:18,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:00:19,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:00:19,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bda178a5bbc0ea6350/lava_callback.json.gz 2025-09-28 04:00:19,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:19,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bda178a5bbc0ea6350 HTTP/1.1" 200 None 2025-09-28 04:00:19,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:19,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:00:19,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:19,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bda178a5bbc0ea6350 HTTP/1.1" 200 None 2025-09-28 04:00:19,092 - lava_callback - INFO - Completed processing callback for node 68d8b1bda178a5bbc0ea6350 INFO: 172.21.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37546 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:00:41,211 - lava_callback - INFO - Received callback for node 68d7f9c38dd6b39fd1aeb7ea INFO: 172.21.0.1:58770 - "POST /node/68d7f9c38dd6b39fd1aeb7ea HTTP/1.0" 202 Accepted 2025-09-28 04:00:41,218 - lava_callback - INFO - Starting async job submit for node 68d7f9c38dd6b39fd1aeb7ea 2025-09-28 04:00:41,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:41,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9c38dd6b39fd1aeb7ea HTTP/1.1" 200 None 2025-09-28 04:00:41,293 - lava_callback - DEBUG - Processing node 68d7f9c38dd6b39fd1aeb7ea: baseline-arm64 2025-09-28 04:00:42,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:00:42,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:00:42,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9c38dd6b39fd1aeb7ea/log.txt.gz 2025-09-28 04:00:42,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:00:42,415 - lava_callback - INFO - Received callback for node 68d8b1cba178a5bbc0ea63b0 INFO: 172.21.0.1:58780 - "POST /node/68d8b1cba178a5bbc0ea63b0 HTTP/1.0" 202 Accepted 2025-09-28 04:00:42,421 - lava_callback - INFO - Starting async job submit for node 68d8b1cba178a5bbc0ea63b0 2025-09-28 04:00:42,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:42,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cba178a5bbc0ea63b0 HTTP/1.1" 200 None 2025-09-28 04:00:42,476 - lava_callback - DEBUG - Processing node 68d8b1cba178a5bbc0ea63b0: baseline-arm-clabbe 2025-09-28 04:00:42,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:00:42,562 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7f9c38dd6b39fd1aeb7ea/lava_callback.json.gz DEBUG: Setup failed for 68d7f9c38dd6b39fd1aeb7ea. Transitting job node result: incomplete -> fail 2025-09-28 04:00:42,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:42,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:00:42,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7f9c38dd6b39fd1aeb7ea HTTP/1.1" 200 None 2025-09-28 04:00:42,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:42,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 04:00:42,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:42,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7f9c38dd6b39fd1aeb7ea HTTP/1.1" 200 None 2025-09-28 04:00:42,816 - lava_callback - INFO - Completed processing callback for node 68d7f9c38dd6b39fd1aeb7ea 2025-09-28 04:00:42,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:00:42,925 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d8b1cba178a5bbc0ea63b0/log.txt.gz 2025-09-28 04:00:42,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:00:43,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:00:43,059 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-68d8b1cba178a5bbc0ea63b0/lava_callback.json.gz 2025-09-28 04:00:43,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:43,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1cba178a5bbc0ea63b0 HTTP/1.1" 200 None 2025-09-28 04:00:43,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:43,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:00:43,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:00:43,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1cba178a5bbc0ea63b0 HTTP/1.1" 200 None 2025-09-28 04:00:43,149 - lava_callback - INFO - Completed processing callback for node 68d8b1cba178a5bbc0ea63b0 INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48770 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:01:11,543 - lava_callback - INFO - Received callback for node 68d8b1c2a178a5bbc0ea6374 INFO: 172.21.0.1:41350 - "POST /node/68d8b1c2a178a5bbc0ea6374 HTTP/1.0" 202 Accepted 2025-09-28 04:01:11,549 - lava_callback - INFO - Starting async job submit for node 68d8b1c2a178a5bbc0ea6374 2025-09-28 04:01:11,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:01:11,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c2a178a5bbc0ea6374 HTTP/1.1" 200 None 2025-09-28 04:01:11,593 - lava_callback - DEBUG - Processing node 68d8b1c2a178a5bbc0ea6374: baseline-arm-baylibre 2025-09-28 04:01:11,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:01:11,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:01:11,998 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c2a178a5bbc0ea6374/log.txt.gz 2025-09-28 04:01:12,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:01:12,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:01:12,119 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-baylibre-68d8b1c2a178a5bbc0ea6374/lava_callback.json.gz 2025-09-28 04:01:12,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:01:12,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1c2a178a5bbc0ea6374 HTTP/1.1" 200 None 2025-09-28 04:01:12,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:01:12,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:01:12,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:01:12,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1c2a178a5bbc0ea6374 HTTP/1.1" 200 None 2025-09-28 04:01:12,223 - lava_callback - INFO - Completed processing callback for node 68d8b1c2a178a5bbc0ea6374 INFO: 172.21.0.1:41360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36000 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:02:19,875 - lava_callback - INFO - Received callback for node 68d8b16ca178a5bbc0ea6315 INFO: 172.21.0.1:35480 - "POST /node/68d8b16ca178a5bbc0ea6315 HTTP/1.0" 202 Accepted 2025-09-28 04:02:19,880 - lava_callback - INFO - Starting async job submit for node 68d8b16ca178a5bbc0ea6315 2025-09-28 04:02:19,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:19,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b16ca178a5bbc0ea6315 HTTP/1.1" 200 None 2025-09-28 04:02:19,912 - lava_callback - DEBUG - Processing node 68d8b16ca178a5bbc0ea6315: baseline-x86 2025-09-28 04:02:20,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:02:20,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:02:20,330 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b16ca178a5bbc0ea6315/log.txt.gz 2025-09-28 04:02:20,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:02:20,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:02:20,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b16ca178a5bbc0ea6315/lava_callback.json.gz 2025-09-28 04:02:20,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:20,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b16ca178a5bbc0ea6315 HTTP/1.1" 200 None 2025-09-28 04:02:20,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:20,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aafca178a5bbc0ea5cd2 HTTP/1.1" 200 None 2025-09-28 04:02:20,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:20,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b16ca178a5bbc0ea6315 HTTP/1.1" 200 None 2025-09-28 04:02:20,583 - lava_callback - INFO - Completed processing callback for node 68d8b16ca178a5bbc0ea6315 INFO: 172.21.0.1:35482 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:02:32,521 - lava_callback - INFO - Received callback for node 68d8a3af7aff7facd9a8d4b0 INFO: 172.21.0.1:57040 - "POST /node/68d8a3af7aff7facd9a8d4b0 HTTP/1.0" 202 Accepted 2025-09-28 04:02:32,542 - lava_callback - INFO - Starting async job submit for node 68d8a3af7aff7facd9a8d4b0 2025-09-28 04:02:33,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:33,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3af7aff7facd9a8d4b0 HTTP/1.1" 200 None 2025-09-28 04:02:33,958 - lava_callback - DEBUG - Processing node 68d8a3af7aff7facd9a8d4b0: ltp-syscalls INFO: 172.21.0.1:57044 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:02:40,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:02:40,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:02:40,983 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-68d8a3af7aff7facd9a8d4b0/log.txt.gz 2025-09-28 04:02:41,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:02:41,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:02:41,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-68d8a3af7aff7facd9a8d4b0/lava_callback.json.gz DEBUG: 68d8a3af7aff7facd9a8d4b0 Transitting job node result: pass -> fail 2025-09-28 04:02:41,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:41,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3af7aff7facd9a8d4b0 HTTP/1.1" 200 None 2025-09-28 04:02:41,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:41,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:02:41,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:02:43,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3af7aff7facd9a8d4b0 HTTP/1.1" 200 None 2025-09-28 04:02:43,385 - lava_callback - INFO - Completed processing callback for node 68d8a3af7aff7facd9a8d4b0 INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59740 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:03:51,438 - lava_callback - INFO - Received callback for node 68d6179130c58dac7d941fcd INFO: 172.21.0.1:58782 - "POST /node/68d6179130c58dac7d941fcd HTTP/1.0" 202 Accepted 2025-09-28 04:03:51,444 - lava_callback - INFO - Starting async job submit for node 68d6179130c58dac7d941fcd 2025-09-28 04:03:51,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:03:51,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179130c58dac7d941fcd HTTP/1.1" 200 None 2025-09-28 04:03:51,483 - lava_callback - DEBUG - Processing node 68d6179130c58dac7d941fcd: ltp-fsx 2025-09-28 04:03:51,810 - lava_callback - ERROR - Error processing callback for node 68d6179130c58dac7d941fcd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-28 04:03:53,057 - lava_callback - INFO - Received callback for node 68d8b1f1a178a5bbc0ea6402 INFO: 172.21.0.1:58794 - "POST /node/68d8b1f1a178a5bbc0ea6402 HTTP/1.0" 202 Accepted 2025-09-28 04:03:53,068 - lava_callback - INFO - Starting async job submit for node 68d8b1f1a178a5bbc0ea6402 2025-09-28 04:03:53,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:03:53,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1f1a178a5bbc0ea6402 HTTP/1.1" 200 None 2025-09-28 04:03:53,126 - lava_callback - DEBUG - Processing node 68d8b1f1a178a5bbc0ea6402: ltp-sched 2025-09-28 04:03:53,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:03:53,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:03:53,839 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-68d8b1f1a178a5bbc0ea6402/log.txt.gz 2025-09-28 04:03:53,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:03:53,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:03:53,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-68d8b1f1a178a5bbc0ea6402/lava_callback.json.gz DEBUG: 68d8b1f1a178a5bbc0ea6402 Transitting job node result: pass -> fail 2025-09-28 04:03:54,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:03:54,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1f1a178a5bbc0ea6402 HTTP/1.1" 200 None 2025-09-28 04:03:54,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:03:54,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:03:54,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:03:54,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1f1a178a5bbc0ea6402 HTTP/1.1" 200 None 2025-09-28 04:03:54,117 - lava_callback - INFO - Completed processing callback for node 68d8b1f1a178a5bbc0ea6402 INFO: 172.21.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49238 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:04:11,321 - lava_callback - INFO - Received callback for node 68d6177c30c58dac7d941fa9 INFO: 172.21.0.1:39756 - "POST /node/68d6177c30c58dac7d941fa9 HTTP/1.0" 202 Accepted 2025-09-28 04:04:11,327 - lava_callback - INFO - Starting async job submit for node 68d6177c30c58dac7d941fa9 2025-09-28 04:04:11,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:04:11,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6177c30c58dac7d941fa9 HTTP/1.1" 200 None 2025-09-28 04:04:11,361 - lava_callback - DEBUG - Processing node 68d6177c30c58dac7d941fa9: baseline-arm-broonie 2025-09-28 04:04:11,574 - lava_callback - ERROR - Error processing callback for node 68d6177c30c58dac7d941fa9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39784 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:04:19,918 - lava_callback - INFO - Received callback for node 68d8aa8ba178a5bbc0ea5c8c INFO: 172.21.0.1:50924 - "POST /node/68d8aa8ba178a5bbc0ea5c8c HTTP/1.0" 202 Accepted 2025-09-28 04:04:19,935 - lava_callback - INFO - Starting async job submit for node 68d8aa8ba178a5bbc0ea5c8c 2025-09-28 04:04:20,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:04:20,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa8ba178a5bbc0ea5c8c HTTP/1.1" 200 None 2025-09-28 04:04:20,067 - lava_callback - DEBUG - Processing node 68d8aa8ba178a5bbc0ea5c8c: baseline-arm 2025-09-28 04:04:20,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:04:20,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:04:20,792 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8aa8ba178a5bbc0ea5c8c/log.txt.gz 2025-09-28 04:04:20,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:04:21,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:04:21,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8aa8ba178a5bbc0ea5c8c/lava_callback.json.gz DEBUG: Setup failed for 68d8aa8ba178a5bbc0ea5c8c. Transitting job node result: incomplete -> fail 2025-09-28 04:04:21,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:04:21,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa8ba178a5bbc0ea5c8c HTTP/1.1" 200 None 2025-09-28 04:04:21,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:04:21,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:04:21,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:04:21,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8aa8ba178a5bbc0ea5c8c HTTP/1.1" 200 None 2025-09-28 04:04:21,211 - lava_callback - INFO - Completed processing callback for node 68d8aa8ba178a5bbc0ea5c8c INFO: 172.21.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:05:10,440 - lava_callback - INFO - Received callback for node 68d7fad88dd6b39fd1aeb7f6 INFO: 172.21.0.1:41904 - "POST /node/68d7fad88dd6b39fd1aeb7f6 HTTP/1.0" 202 Accepted 2025-09-28 04:05:10,452 - lava_callback - INFO - Starting async job submit for node 68d7fad88dd6b39fd1aeb7f6 2025-09-28 04:05:10,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:10,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fad88dd6b39fd1aeb7f6 HTTP/1.1" 200 None 2025-09-28 04:05:10,569 - lava_callback - DEBUG - Processing node 68d7fad88dd6b39fd1aeb7f6: baseline-arm64 2025-09-28 04:05:12,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:05:12,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:05:12,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fad88dd6b39fd1aeb7f6/log.txt.gz 2025-09-28 04:05:12,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:05:12,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:05:12,502 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fad88dd6b39fd1aeb7f6/lava_callback.json.gz DEBUG: Setup failed for 68d7fad88dd6b39fd1aeb7f6. Transitting job node result: incomplete -> fail 2025-09-28 04:05:12,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:12,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fad88dd6b39fd1aeb7f6 HTTP/1.1" 200 None 2025-09-28 04:05:12,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:12,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d79eda8dd6b39fd1aea2de HTTP/1.1" 200 None 2025-09-28 04:05:12,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:12,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fad88dd6b39fd1aeb7f6 HTTP/1.1" 200 None 2025-09-28 04:05:12,703 - lava_callback - INFO - Completed processing callback for node 68d7fad88dd6b39fd1aeb7f6 INFO: 172.21.0.1:41912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41566 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:05:30,074 - lava_callback - INFO - Received callback for node 68d7fad98dd6b39fd1aeb7f9 INFO: 172.21.0.1:55560 - "POST /node/68d7fad98dd6b39fd1aeb7f9 HTTP/1.0" 202 Accepted 2025-09-28 04:05:30,091 - lava_callback - INFO - Starting async job submit for node 68d7fad98dd6b39fd1aeb7f9 2025-09-28 04:05:30,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:30,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fad98dd6b39fd1aeb7f9 HTTP/1.1" 200 None 2025-09-28 04:05:30,163 - lava_callback - DEBUG - Processing node 68d7fad98dd6b39fd1aeb7f9: baseline-arm64 2025-09-28 04:05:31,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:05:31,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:05:31,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fad98dd6b39fd1aeb7f9/log.txt.gz 2025-09-28 04:05:31,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:05:32,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:05:32,182 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fad98dd6b39fd1aeb7f9/lava_callback.json.gz DEBUG: Setup failed for 68d7fad98dd6b39fd1aeb7f9. Transitting job node result: incomplete -> fail 2025-09-28 04:05:32,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:32,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fad98dd6b39fd1aeb7f9 HTTP/1.1" 200 None 2025-09-28 04:05:32,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:32,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d79eda8dd6b39fd1aea2de HTTP/1.1" 200 None 2025-09-28 04:05:32,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:05:32,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fad98dd6b39fd1aeb7f9 HTTP/1.1" 200 None 2025-09-28 04:05:32,421 - lava_callback - INFO - Completed processing callback for node 68d7fad98dd6b39fd1aeb7f9 INFO: 172.21.0.1:55562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34226 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:08:31,625 - lava_callback - INFO - Received callback for node 68d6179330c58dac7d941fd0 INFO: 172.21.0.1:50650 - "POST /node/68d6179330c58dac7d941fd0 HTTP/1.0" 202 Accepted 2025-09-28 04:08:31,632 - lava_callback - INFO - Starting async job submit for node 68d6179330c58dac7d941fd0 2025-09-28 04:08:31,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:08:31,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179330c58dac7d941fd0 HTTP/1.1" 200 None 2025-09-28 04:08:31,695 - lava_callback - DEBUG - Processing node 68d6179330c58dac7d941fd0: ltp-input 2025-09-28 04:08:32,079 - lava_callback - ERROR - Error processing callback for node 68d6179330c58dac7d941fd0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37438 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:09:01,021 - lava_callback - INFO - Received callback for node 68d8a3af7aff7facd9a8d4b3 INFO: 172.21.0.1:40888 - "POST /node/68d8a3af7aff7facd9a8d4b3 HTTP/1.0" 202 Accepted 2025-09-28 04:09:01,029 - lava_callback - INFO - Starting async job submit for node 68d8a3af7aff7facd9a8d4b3 2025-09-28 04:09:01,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:09:01,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3af7aff7facd9a8d4b3 HTTP/1.1" 200 None 2025-09-28 04:09:01,084 - lava_callback - DEBUG - Processing node 68d8a3af7aff7facd9a8d4b3: ltp-syscalls-ipc INFO: 172.21.0.1:40894 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:09:01,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:09:02,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:09:02,097 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8a3af7aff7facd9a8d4b3/log.txt.gz 2025-09-28 04:09:02,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:09:02,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:09:02,236 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8a3af7aff7facd9a8d4b3/lava_callback.json.gz 2025-09-28 04:09:02,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:09:02,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a3af7aff7facd9a8d4b3 HTTP/1.1" 200 None 2025-09-28 04:09:02,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:09:02,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:09:02,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:09:02,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a3af7aff7facd9a8d4b3 HTTP/1.1" 200 None 2025-09-28 04:09:02,378 - lava_callback - INFO - Completed processing callback for node 68d8a3af7aff7facd9a8d4b3 INFO: 172.21.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33258 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:10:44,303 - lava_callback - INFO - Received callback for node 68d8b0e4a178a5bbc0ea6304 INFO: 172.21.0.1:54378 - "POST /node/68d8b0e4a178a5bbc0ea6304 HTTP/1.0" 202 Accepted 2025-09-28 04:10:44,311 - lava_callback - INFO - Starting async job submit for node 68d8b0e4a178a5bbc0ea6304 2025-09-28 04:10:44,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:10:44,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b0e4a178a5bbc0ea6304 HTTP/1.1" 200 None 2025-09-28 04:10:44,364 - lava_callback - DEBUG - Processing node 68d8b0e4a178a5bbc0ea6304: baseline-arm 2025-09-28 04:10:44,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:10:44,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:10:44,854 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b0e4a178a5bbc0ea6304/log.txt.gz 2025-09-28 04:10:44,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:10:45,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:10:45,031 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b0e4a178a5bbc0ea6304/lava_callback.json.gz DEBUG: Setup failed for 68d8b0e4a178a5bbc0ea6304. Transitting job node result: incomplete -> fail 2025-09-28 04:10:45,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:10:45,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b0e4a178a5bbc0ea6304 HTTP/1.1" 200 None 2025-09-28 04:10:45,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:10:45,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:10:45,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:10:45,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b0e4a178a5bbc0ea6304 HTTP/1.1" 200 None 2025-09-28 04:10:45,157 - lava_callback - INFO - Completed processing callback for node 68d8b0e4a178a5bbc0ea6304 INFO: 172.21.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37106 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:13:11,843 - lava_callback - INFO - Received callback for node 68d6179530c58dac7d941fd3 INFO: 172.21.0.1:55390 - "POST /node/68d6179530c58dac7d941fd3 HTTP/1.0" 202 Accepted 2025-09-28 04:13:11,850 - lava_callback - INFO - Starting async job submit for node 68d6179530c58dac7d941fd3 2025-09-28 04:13:11,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:13:11,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179530c58dac7d941fd3 HTTP/1.1" 200 None 2025-09-28 04:13:11,907 - lava_callback - DEBUG - Processing node 68d6179530c58dac7d941fd3: ltp-ipc 2025-09-28 04:13:12,259 - lava_callback - ERROR - Error processing callback for node 68d6179530c58dac7d941fd3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:14:24,052 - lava_callback - INFO - Received callback for node 68d8b17ea178a5bbc0ea6323 INFO: 172.21.0.1:58118 - "POST /node/68d8b17ea178a5bbc0ea6323 HTTP/1.0" 202 Accepted 2025-09-28 04:14:24,059 - lava_callback - INFO - Starting async job submit for node 68d8b17ea178a5bbc0ea6323 2025-09-28 04:14:24,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:24,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b17ea178a5bbc0ea6323 HTTP/1.1" 200 None 2025-09-28 04:14:24,122 - lava_callback - DEBUG - Processing node 68d8b17ea178a5bbc0ea6323: baseline-x86 2025-09-28 04:14:24,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:24,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:24,972 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b17ea178a5bbc0ea6323/log.txt.gz 2025-09-28 04:14:24,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:25,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:25,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b17ea178a5bbc0ea6323/lava_callback.json.gz DEBUG: Setup failed for 68d8b17ea178a5bbc0ea6323. Transitting job node result: incomplete -> fail 2025-09-28 04:14:25,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:25,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b17ea178a5bbc0ea6323 HTTP/1.1" 200 None 2025-09-28 04:14:25,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:25,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aafca178a5bbc0ea5cd2 HTTP/1.1" 200 None 2025-09-28 04:14:25,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:25,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b17ea178a5bbc0ea6323 HTTP/1.1" 200 None 2025-09-28 04:14:25,341 - lava_callback - INFO - Completed processing callback for node 68d8b17ea178a5bbc0ea6323 INFO: 172.21.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:14:44,020 - lava_callback - INFO - Received callback for node 68d8b5f7a178a5bbc0ea6ad8 INFO: 172.21.0.1:51090 - "POST /node/68d8b5f7a178a5bbc0ea6ad8 HTTP/1.0" 202 Accepted 2025-09-28 04:14:44,027 - lava_callback - INFO - Starting async job submit for node 68d8b5f7a178a5bbc0ea6ad8 2025-09-28 04:14:44,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:44,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f7a178a5bbc0ea6ad8 HTTP/1.1" 200 None 2025-09-28 04:14:44,072 - lava_callback - DEBUG - Processing node 68d8b5f7a178a5bbc0ea6ad8: baseline-arm64-foundriesio 2025-09-28 04:14:44,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:44,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:44,580 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d8b5f7a178a5bbc0ea6ad8/log.txt.gz 2025-09-28 04:14:44,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:44,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:44,729 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d8b5f7a178a5bbc0ea6ad8/lava_callback.json.gz 2025-09-28 04:14:44,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:44,760 - lava_callback - INFO - Received callback for node 68d8b1bea178a5bbc0ea6359 INFO: 172.21.0.1:51102 - "POST /node/68d8b1bea178a5bbc0ea6359 HTTP/1.0" 202 Accepted 2025-09-28 04:14:44,766 - lava_callback - INFO - Starting async job submit for node 68d8b1bea178a5bbc0ea6359 2025-09-28 04:14:44,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:44,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f7a178a5bbc0ea6ad8 HTTP/1.1" 200 None 2025-09-28 04:14:44,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:44,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bea178a5bbc0ea6359 HTTP/1.1" 200 None 2025-09-28 04:14:44,817 - lava_callback - DEBUG - Processing node 68d8b1bea178a5bbc0ea6359: baseline-arm 2025-09-28 04:14:44,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:14:44,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:44,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5f7a178a5bbc0ea6ad8 HTTP/1.1" 200 None 2025-09-28 04:14:44,952 - lava_callback - INFO - Completed processing callback for node 68d8b5f7a178a5bbc0ea6ad8 2025-09-28 04:14:45,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:45,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:45,439 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bea178a5bbc0ea6359/log.txt.gz 2025-09-28 04:14:45,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:45,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:45,580 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bea178a5bbc0ea6359/lava_callback.json.gz DEBUG: Setup failed for 68d8b1bea178a5bbc0ea6359. Transitting job node result: incomplete -> fail 2025-09-28 04:14:45,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:45,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bea178a5bbc0ea6359 HTTP/1.1" 200 None 2025-09-28 04:14:45,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:45,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:14:45,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:45,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bea178a5bbc0ea6359 HTTP/1.1" 200 None 2025-09-28 04:14:45,697 - lava_callback - INFO - Completed processing callback for node 68d8b1bea178a5bbc0ea6359 2025-09-28 04:14:46,180 - lava_callback - INFO - Received callback for node 68d8b5f5a178a5bbc0ea6acc INFO: 172.21.0.1:51104 - "POST /node/68d8b5f5a178a5bbc0ea6acc HTTP/1.0" 202 Accepted 2025-09-28 04:14:46,186 - lava_callback - INFO - Starting async job submit for node 68d8b5f5a178a5bbc0ea6acc 2025-09-28 04:14:46,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:46,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f5a178a5bbc0ea6acc HTTP/1.1" 200 None 2025-09-28 04:14:46,218 - lava_callback - DEBUG - Processing node 68d8b5f5a178a5bbc0ea6acc: baseline-arm64-pengutronix 2025-09-28 04:14:46,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:46,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:46,521 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d8b5f5a178a5bbc0ea6acc/log.txt.gz 2025-09-28 04:14:46,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:51106 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:14:46,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:46,701 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d8b5f5a178a5bbc0ea6acc/lava_callback.json.gz 2025-09-28 04:14:46,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:46,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f5a178a5bbc0ea6acc HTTP/1.1" 200 None 2025-09-28 04:14:46,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:46,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:14:46,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:46,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5f5a178a5bbc0ea6acc HTTP/1.1" 200 None 2025-09-28 04:14:46,779 - lava_callback - INFO - Completed processing callback for node 68d8b5f5a178a5bbc0ea6acc 2025-09-28 04:14:51,848 - lava_callback - INFO - Received callback for node 68d8b5f6a178a5bbc0ea6ad5 INFO: 172.21.0.1:53988 - "POST /node/68d8b5f6a178a5bbc0ea6ad5 HTTP/1.0" 202 Accepted 2025-09-28 04:14:51,854 - lava_callback - INFO - Starting async job submit for node 68d8b5f6a178a5bbc0ea6ad5 2025-09-28 04:14:51,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:51,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f6a178a5bbc0ea6ad5 HTTP/1.1" 200 None 2025-09-28 04:14:51,887 - lava_callback - DEBUG - Processing node 68d8b5f6a178a5bbc0ea6ad5: baseline-arm64-foundriesio 2025-09-28 04:14:52,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:52,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:52,224 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d8b5f6a178a5bbc0ea6ad5/log.txt.gz 2025-09-28 04:14:52,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:52,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:52,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-68d8b5f6a178a5bbc0ea6ad5/lava_callback.json.gz 2025-09-28 04:14:52,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:52,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f6a178a5bbc0ea6ad5 HTTP/1.1" 200 None 2025-09-28 04:14:52,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:52,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:14:52,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:52,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5f6a178a5bbc0ea6ad5 HTTP/1.1" 200 None 2025-09-28 04:14:52,455 - lava_callback - INFO - Completed processing callback for node 68d8b5f6a178a5bbc0ea6ad5 INFO: 172.21.0.1:53998 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:14:58,748 - lava_callback - INFO - Received callback for node 68d8b5f4a178a5bbc0ea6ac6 INFO: 172.21.0.1:54014 - "POST /node/68d8b5f4a178a5bbc0ea6ac6 HTTP/1.0" 202 Accepted 2025-09-28 04:14:58,754 - lava_callback - INFO - Starting async job submit for node 68d8b5f4a178a5bbc0ea6ac6 2025-09-28 04:14:58,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:58,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f4a178a5bbc0ea6ac6 HTTP/1.1" 200 None 2025-09-28 04:14:58,789 - lava_callback - DEBUG - Processing node 68d8b5f4a178a5bbc0ea6ac6: baseline-arm64-kontron 2025-09-28 04:14:59,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:59,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:59,229 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b5f4a178a5bbc0ea6ac6/log.txt.gz 2025-09-28 04:14:59,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:14:59,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:14:59,372 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b5f4a178a5bbc0ea6ac6/lava_callback.json.gz 2025-09-28 04:14:59,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:59,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f4a178a5bbc0ea6ac6 HTTP/1.1" 200 None 2025-09-28 04:14:59,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:59,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:14:59,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:14:59,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5f4a178a5bbc0ea6ac6 HTTP/1.1" 200 None 2025-09-28 04:14:59,479 - lava_callback - INFO - Completed processing callback for node 68d8b5f4a178a5bbc0ea6ac6 INFO: 172.21.0.1:45742 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:15:04,773 - lava_callback - INFO - Received callback for node 68d8b5eaa178a5bbc0ea6a8d INFO: 172.21.0.1:45744 - "POST /node/68d8b5eaa178a5bbc0ea6a8d HTTP/1.0" 202 Accepted 2025-09-28 04:15:04,778 - lava_callback - INFO - Starting async job submit for node 68d8b5eaa178a5bbc0ea6a8d 2025-09-28 04:15:04,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:04,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eaa178a5bbc0ea6a8d HTTP/1.1" 200 None 2025-09-28 04:15:04,812 - lava_callback - DEBUG - Processing node 68d8b5eaa178a5bbc0ea6a8d: baseline-arm64-broonie 2025-09-28 04:15:04,924 - lava_callback - INFO - Received callback for node 68d8b5dea178a5bbc0ea6a36 INFO: 172.21.0.1:45758 - "POST /node/68d8b5dea178a5bbc0ea6a36 HTTP/1.0" 202 Accepted 2025-09-28 04:15:04,952 - lava_callback - INFO - Starting async job submit for node 68d8b5dea178a5bbc0ea6a36 2025-09-28 04:15:04,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:05,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dea178a5bbc0ea6a36 HTTP/1.1" 200 None 2025-09-28 04:15:05,064 - lava_callback - DEBUG - Processing node 68d8b5dea178a5bbc0ea6a36: baseline-arm64 2025-09-28 04:15:05,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:05,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:05,353 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eaa178a5bbc0ea6a8d/log.txt.gz 2025-09-28 04:15:05,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:05,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:05,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:05,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:05,508 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dea178a5bbc0ea6a36/log.txt.gz 2025-09-28 04:15:05,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eaa178a5bbc0ea6a8d/lava_callback.json.gz 2025-09-28 04:15:05,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:05,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:05,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eaa178a5bbc0ea6a8d HTTP/1.1" 200 None 2025-09-28 04:15:05,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:05,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:05,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:05,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eaa178a5bbc0ea6a8d HTTP/1.1" 200 None 2025-09-28 04:15:05,602 - lava_callback - INFO - Completed processing callback for node 68d8b5eaa178a5bbc0ea6a8d 2025-09-28 04:15:05,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:05,708 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dea178a5bbc0ea6a36/lava_callback.json.gz 2025-09-28 04:15:05,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:05,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dea178a5bbc0ea6a36 HTTP/1.1" 200 None 2025-09-28 04:15:05,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:05,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:05,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:05,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dea178a5bbc0ea6a36 HTTP/1.1" 200 None 2025-09-28 04:15:05,798 - lava_callback - INFO - Completed processing callback for node 68d8b5dea178a5bbc0ea6a36 INFO: 172.21.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45782 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:15:12,470 - lava_callback - INFO - Received callback for node 68d8b5e1a178a5bbc0ea6a4e INFO: 172.21.0.1:57606 - "POST /node/68d8b5e1a178a5bbc0ea6a4e HTTP/1.0" 202 Accepted 2025-09-28 04:15:12,476 - lava_callback - INFO - Starting async job submit for node 68d8b5e1a178a5bbc0ea6a4e 2025-09-28 04:15:12,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:12,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e1a178a5bbc0ea6a4e HTTP/1.1" 200 None 2025-09-28 04:15:12,510 - lava_callback - DEBUG - Processing node 68d8b5e1a178a5bbc0ea6a4e: baseline-arm64 2025-09-28 04:15:12,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:12,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:12,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e1a178a5bbc0ea6a4e/log.txt.gz 2025-09-28 04:15:12,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:13,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:13,053 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e1a178a5bbc0ea6a4e/lava_callback.json.gz 2025-09-28 04:15:13,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:13,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e1a178a5bbc0ea6a4e HTTP/1.1" 200 None 2025-09-28 04:15:13,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:13,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:13,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:13,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e1a178a5bbc0ea6a4e HTTP/1.1" 200 None 2025-09-28 04:15:13,137 - lava_callback - INFO - Completed processing callback for node 68d8b5e1a178a5bbc0ea6a4e 2025-09-28 04:15:14,613 - lava_callback - INFO - Received callback for node 68d8b5dfa178a5bbc0ea6a3c INFO: 172.21.0.1:57614 - "POST /node/68d8b5dfa178a5bbc0ea6a3c HTTP/1.0" 202 Accepted 2025-09-28 04:15:14,619 - lava_callback - INFO - Starting async job submit for node 68d8b5dfa178a5bbc0ea6a3c 2025-09-28 04:15:14,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:14,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dfa178a5bbc0ea6a3c HTTP/1.1" 200 None 2025-09-28 04:15:14,662 - lava_callback - DEBUG - Processing node 68d8b5dfa178a5bbc0ea6a3c: baseline-arm64 2025-09-28 04:15:14,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:15,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:15,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dfa178a5bbc0ea6a3c/log.txt.gz 2025-09-28 04:15:15,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:57616 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 04:15:15,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:15,205 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dfa178a5bbc0ea6a3c/lava_callback.json.gz 2025-09-28 04:15:15,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:15,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dfa178a5bbc0ea6a3c HTTP/1.1" 200 None 2025-09-28 04:15:15,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:15,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:15,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:15,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dfa178a5bbc0ea6a3c HTTP/1.1" 200 None 2025-09-28 04:15:15,293 - lava_callback - INFO - Completed processing callback for node 68d8b5dfa178a5bbc0ea6a3c INFO: 172.21.0.1:57626 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:15:18,779 - lava_callback - INFO - Received callback for node 68d8b5f3a178a5bbc0ea6ac3 INFO: 172.21.0.1:57634 - "POST /node/68d8b5f3a178a5bbc0ea6ac3 HTTP/1.0" 202 Accepted 2025-09-28 04:15:18,789 - lava_callback - INFO - Starting async job submit for node 68d8b5f3a178a5bbc0ea6ac3 2025-09-28 04:15:18,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f3a178a5bbc0ea6ac3 HTTP/1.1" 200 None 2025-09-28 04:15:18,864 - lava_callback - DEBUG - Processing node 68d8b5f3a178a5bbc0ea6ac3: baseline-arm64-kontron 2025-09-28 04:15:19,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:19,239 - lava_callback - INFO - Received callback for node 68d8b5efa178a5bbc0ea6aab INFO: 172.21.0.1:57644 - "POST /node/68d8b5efa178a5bbc0ea6aab HTTP/1.0" 202 Accepted 2025-09-28 04:15:19,248 - lava_callback - INFO - Starting async job submit for node 68d8b5efa178a5bbc0ea6aab 2025-09-28 04:15:19,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:19,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:19,276 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b5f3a178a5bbc0ea6ac3/log.txt.gz 2025-09-28 04:15:19,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:19,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5efa178a5bbc0ea6aab HTTP/1.1" 200 None 2025-09-28 04:15:19,291 - lava_callback - DEBUG - Processing node 68d8b5efa178a5bbc0ea6aab: baseline-arm64-clabbe 2025-09-28 04:15:19,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:19,476 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b5f3a178a5bbc0ea6ac3/lava_callback.json.gz 2025-09-28 04:15:19,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:19,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f3a178a5bbc0ea6ac3 HTTP/1.1" 200 None 2025-09-28 04:15:19,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:19,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:19,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:19,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:19,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5f3a178a5bbc0ea6ac3 HTTP/1.1" 200 None 2025-09-28 04:15:19,740 - lava_callback - INFO - Completed processing callback for node 68d8b5f3a178a5bbc0ea6ac3 2025-09-28 04:15:19,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:19,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d8b5efa178a5bbc0ea6aab/log.txt.gz 2025-09-28 04:15:19,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:20,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:20,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d8b5efa178a5bbc0ea6aab/lava_callback.json.gz 2025-09-28 04:15:20,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:20,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5efa178a5bbc0ea6aab HTTP/1.1" 200 None 2025-09-28 04:15:20,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:20,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:20,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:20,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5efa178a5bbc0ea6aab HTTP/1.1" 200 None 2025-09-28 04:15:20,119 - lava_callback - INFO - Completed processing callback for node 68d8b5efa178a5bbc0ea6aab 2025-09-28 04:15:21,124 - lava_callback - INFO - Received callback for node 68d8b5e7a178a5bbc0ea6a78 INFO: 172.21.0.1:60736 - "POST /node/68d8b5e7a178a5bbc0ea6a78 HTTP/1.0" 202 Accepted 2025-09-28 04:15:21,130 - lava_callback - INFO - Starting async job submit for node 68d8b5e7a178a5bbc0ea6a78 2025-09-28 04:15:21,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:21,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e7a178a5bbc0ea6a78 HTTP/1.1" 200 None 2025-09-28 04:15:21,166 - lava_callback - DEBUG - Processing node 68d8b5e7a178a5bbc0ea6a78: baseline-arm64-broonie 2025-09-28 04:15:21,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:21,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:21,519 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e7a178a5bbc0ea6a78/log.txt.gz 2025-09-28 04:15:21,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:21,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:21,655 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e7a178a5bbc0ea6a78/lava_callback.json.gz 2025-09-28 04:15:21,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:21,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e7a178a5bbc0ea6a78 HTTP/1.1" 200 None 2025-09-28 04:15:21,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:21,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:21,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:21,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e7a178a5bbc0ea6a78 HTTP/1.1" 200 None 2025-09-28 04:15:21,766 - lava_callback - INFO - Completed processing callback for node 68d8b5e7a178a5bbc0ea6a78 2025-09-28 04:15:22,871 - lava_callback - INFO - Received callback for node 68d8b5dda178a5bbc0ea6a30 INFO: 172.21.0.1:60740 - "POST /node/68d8b5dda178a5bbc0ea6a30 HTTP/1.0" 202 Accepted 2025-09-28 04:15:22,880 - lava_callback - INFO - Starting async job submit for node 68d8b5dda178a5bbc0ea6a30 2025-09-28 04:15:22,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:22,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dda178a5bbc0ea6a30 HTTP/1.1" 200 None 2025-09-28 04:15:22,925 - lava_callback - DEBUG - Processing node 68d8b5dda178a5bbc0ea6a30: baseline-arm64 2025-09-28 04:15:23,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:23,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:23,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dda178a5bbc0ea6a30/log.txt.gz 2025-09-28 04:15:23,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:23,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:23,489 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dda178a5bbc0ea6a30/lava_callback.json.gz 2025-09-28 04:15:23,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:23,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dda178a5bbc0ea6a30 HTTP/1.1" 200 None 2025-09-28 04:15:23,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:23,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:23,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:23,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dda178a5bbc0ea6a30 HTTP/1.1" 200 None 2025-09-28 04:15:23,587 - lava_callback - INFO - Completed processing callback for node 68d8b5dda178a5bbc0ea6a30 2025-09-28 04:15:25,057 - lava_callback - INFO - Received callback for node 68d8b5eba178a5bbc0ea6a99 INFO: 172.21.0.1:60754 - "POST /node/68d8b5eba178a5bbc0ea6a99 HTTP/1.0" 202 Accepted 2025-09-28 04:15:25,064 - lava_callback - INFO - Starting async job submit for node 68d8b5eba178a5bbc0ea6a99 2025-09-28 04:15:25,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:25,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eba178a5bbc0ea6a99 HTTP/1.1" 200 None 2025-09-28 04:15:25,101 - lava_callback - DEBUG - Processing node 68d8b5eba178a5bbc0ea6a99: baseline-arm64-broonie 2025-09-28 04:15:25,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:25,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:25,504 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eba178a5bbc0ea6a99/log.txt.gz 2025-09-28 04:15:25,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:25,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:25,657 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eba178a5bbc0ea6a99/lava_callback.json.gz 2025-09-28 04:15:25,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:25,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eba178a5bbc0ea6a99 HTTP/1.1" 200 None 2025-09-28 04:15:25,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:25,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:25,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:25,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eba178a5bbc0ea6a99 HTTP/1.1" 200 None 2025-09-28 04:15:25,756 - lava_callback - INFO - Completed processing callback for node 68d8b5eba178a5bbc0ea6a99 INFO: 172.21.0.1:60756 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:15:28,509 - lava_callback - INFO - Received callback for node 68d8b5dda178a5bbc0ea6a33 INFO: 172.21.0.1:60770 - "POST /node/68d8b5dda178a5bbc0ea6a33 HTTP/1.0" 202 Accepted 2025-09-28 04:15:28,517 - lava_callback - INFO - Starting async job submit for node 68d8b5dda178a5bbc0ea6a33 2025-09-28 04:15:28,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:28,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dda178a5bbc0ea6a33 HTTP/1.1" 200 None 2025-09-28 04:15:28,558 - lava_callback - DEBUG - Processing node 68d8b5dda178a5bbc0ea6a33: baseline-arm64 2025-09-28 04:15:28,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:29,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:29,125 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dda178a5bbc0ea6a33/log.txt.gz 2025-09-28 04:15:29,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:29,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:29,247 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dda178a5bbc0ea6a33/lava_callback.json.gz 2025-09-28 04:15:29,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:29,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dda178a5bbc0ea6a33 HTTP/1.1" 200 None 2025-09-28 04:15:29,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:29,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:29,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:29,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dda178a5bbc0ea6a33 HTTP/1.1" 200 None 2025-09-28 04:15:29,347 - lava_callback - INFO - Completed processing callback for node 68d8b5dda178a5bbc0ea6a33 2025-09-28 04:15:31,294 - lava_callback - INFO - Received callback for node 68d8b5dba178a5bbc0ea6a21 INFO: 172.21.0.1:40184 - "POST /node/68d8b5dba178a5bbc0ea6a21 HTTP/1.0" 202 Accepted 2025-09-28 04:15:31,300 - lava_callback - INFO - Starting async job submit for node 68d8b5dba178a5bbc0ea6a21 2025-09-28 04:15:31,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:31,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dba178a5bbc0ea6a21 HTTP/1.1" 200 None 2025-09-28 04:15:31,346 - lava_callback - DEBUG - Processing node 68d8b5dba178a5bbc0ea6a21: baseline-arm64 2025-09-28 04:15:31,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:31,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:31,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dba178a5bbc0ea6a21/log.txt.gz 2025-09-28 04:15:31,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:32,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:32,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dba178a5bbc0ea6a21/lava_callback.json.gz 2025-09-28 04:15:32,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:32,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dba178a5bbc0ea6a21 HTTP/1.1" 200 None 2025-09-28 04:15:32,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:32,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:32,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:32,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dba178a5bbc0ea6a21 HTTP/1.1" 200 None 2025-09-28 04:15:32,256 - lava_callback - INFO - Completed processing callback for node 68d8b5dba178a5bbc0ea6a21 2025-09-28 04:15:32,797 - lava_callback - INFO - Received callback for node 68d8b5e9a178a5bbc0ea6a84 INFO: 172.21.0.1:40194 - "POST /node/68d8b5e9a178a5bbc0ea6a84 HTTP/1.0" 202 Accepted 2025-09-28 04:15:32,803 - lava_callback - INFO - Starting async job submit for node 68d8b5e9a178a5bbc0ea6a84 2025-09-28 04:15:32,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:32,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e9a178a5bbc0ea6a84 HTTP/1.1" 200 None 2025-09-28 04:15:32,845 - lava_callback - DEBUG - Processing node 68d8b5e9a178a5bbc0ea6a84: baseline-arm64-broonie 2025-09-28 04:15:33,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:33,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:33,287 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e9a178a5bbc0ea6a84/log.txt.gz 2025-09-28 04:15:33,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:33,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:33,418 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e9a178a5bbc0ea6a84/lava_callback.json.gz 2025-09-28 04:15:33,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:33,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e9a178a5bbc0ea6a84 HTTP/1.1" 200 None 2025-09-28 04:15:33,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:33,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:33,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:33,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e9a178a5bbc0ea6a84 HTTP/1.1" 200 None 2025-09-28 04:15:33,502 - lava_callback - INFO - Completed processing callback for node 68d8b5e9a178a5bbc0ea6a84 2025-09-28 04:15:34,207 - lava_callback - INFO - Received callback for node 68d8b5e9a178a5bbc0ea6a87 INFO: 172.21.0.1:40202 - "POST /node/68d8b5e9a178a5bbc0ea6a87 HTTP/1.0" 202 Accepted 2025-09-28 04:15:34,214 - lava_callback - INFO - Starting async job submit for node 68d8b5e9a178a5bbc0ea6a87 2025-09-28 04:15:34,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:34,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e9a178a5bbc0ea6a87 HTTP/1.1" 200 None 2025-09-28 04:15:34,281 - lava_callback - DEBUG - Processing node 68d8b5e9a178a5bbc0ea6a87: baseline-arm64-broonie 2025-09-28 04:15:34,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:34,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:34,732 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e9a178a5bbc0ea6a87/log.txt.gz 2025-09-28 04:15:34,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:34,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:34,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e9a178a5bbc0ea6a87/lava_callback.json.gz 2025-09-28 04:15:34,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:34,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e9a178a5bbc0ea6a87 HTTP/1.1" 200 None 2025-09-28 04:15:34,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:34,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:34,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:34,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e9a178a5bbc0ea6a87 HTTP/1.1" 200 None 2025-09-28 04:15:34,977 - lava_callback - INFO - Completed processing callback for node 68d8b5e9a178a5bbc0ea6a87 2025-09-28 04:15:35,642 - lava_callback - INFO - Received callback for node 68d8b5eca178a5bbc0ea6a9f INFO: 172.21.0.1:40216 - "POST /node/68d8b5eca178a5bbc0ea6a9f HTTP/1.0" 202 Accepted 2025-09-28 04:15:35,652 - lava_callback - INFO - Starting async job submit for node 68d8b5eca178a5bbc0ea6a9f 2025-09-28 04:15:35,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:35,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eca178a5bbc0ea6a9f HTTP/1.1" 200 None 2025-09-28 04:15:35,735 - lava_callback - DEBUG - Processing node 68d8b5eca178a5bbc0ea6a9f: baseline-arm64-broonie 2025-09-28 04:15:36,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:36,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:36,115 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eca178a5bbc0ea6a9f/log.txt.gz 2025-09-28 04:15:36,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:36,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:36,293 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eca178a5bbc0ea6a9f/lava_callback.json.gz 2025-09-28 04:15:36,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:36,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eca178a5bbc0ea6a9f HTTP/1.1" 200 None 2025-09-28 04:15:36,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:36,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:36,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:36,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eca178a5bbc0ea6a9f HTTP/1.1" 200 None 2025-09-28 04:15:36,445 - lava_callback - INFO - Completed processing callback for node 68d8b5eca178a5bbc0ea6a9f INFO: 172.21.0.1:40226 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:15:39,248 - lava_callback - INFO - Received callback for node 68d8b5e1a178a5bbc0ea6a4b INFO: 172.21.0.1:40232 - "POST /node/68d8b5e1a178a5bbc0ea6a4b HTTP/1.0" 202 Accepted 2025-09-28 04:15:39,255 - lava_callback - INFO - Starting async job submit for node 68d8b5e1a178a5bbc0ea6a4b 2025-09-28 04:15:39,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:39,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e1a178a5bbc0ea6a4b HTTP/1.1" 200 None 2025-09-28 04:15:39,300 - lava_callback - DEBUG - Processing node 68d8b5e1a178a5bbc0ea6a4b: baseline-arm64 2025-09-28 04:15:39,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:39,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:39,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e1a178a5bbc0ea6a4b/log.txt.gz 2025-09-28 04:15:39,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:39,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:39,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e1a178a5bbc0ea6a4b/lava_callback.json.gz 2025-09-28 04:15:40,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:40,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e1a178a5bbc0ea6a4b HTTP/1.1" 200 None 2025-09-28 04:15:40,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:40,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:40,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:40,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e1a178a5bbc0ea6a4b HTTP/1.1" 200 None 2025-09-28 04:15:40,080 - lava_callback - INFO - Completed processing callback for node 68d8b5e1a178a5bbc0ea6a4b 2025-09-28 04:15:41,781 - lava_callback - INFO - Received callback for node 68d8b5eca178a5bbc0ea6a9c INFO: 172.21.0.1:43176 - "POST /node/68d8b5eca178a5bbc0ea6a9c HTTP/1.0" 202 Accepted 2025-09-28 04:15:41,790 - lava_callback - INFO - Starting async job submit for node 68d8b5eca178a5bbc0ea6a9c 2025-09-28 04:15:41,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:41,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eca178a5bbc0ea6a9c HTTP/1.1" 200 None 2025-09-28 04:15:41,828 - lava_callback - DEBUG - Processing node 68d8b5eca178a5bbc0ea6a9c: baseline-arm64-broonie 2025-09-28 04:15:42,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:42,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:42,299 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eca178a5bbc0ea6a9c/log.txt.gz 2025-09-28 04:15:42,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:42,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:42,552 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eca178a5bbc0ea6a9c/lava_callback.json.gz 2025-09-28 04:15:42,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:42,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eca178a5bbc0ea6a9c HTTP/1.1" 200 None 2025-09-28 04:15:42,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:42,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:42,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:42,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eca178a5bbc0ea6a9c HTTP/1.1" 200 None 2025-09-28 04:15:42,652 - lava_callback - INFO - Completed processing callback for node 68d8b5eca178a5bbc0ea6a9c 2025-09-28 04:15:45,620 - lava_callback - INFO - Received callback for node 68d8b5f2a178a5bbc0ea6ac0 INFO: 172.21.0.1:43188 - "POST /node/68d8b5f2a178a5bbc0ea6ac0 HTTP/1.0" 202 Accepted 2025-09-28 04:15:45,626 - lava_callback - INFO - Starting async job submit for node 68d8b5f2a178a5bbc0ea6ac0 2025-09-28 04:15:45,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:45,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f2a178a5bbc0ea6ac0 HTTP/1.1" 200 None 2025-09-28 04:15:45,663 - lava_callback - DEBUG - Processing node 68d8b5f2a178a5bbc0ea6ac0: baseline-arm64-clabbe 2025-09-28 04:15:45,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:46,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:46,015 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d8b5f2a178a5bbc0ea6ac0/log.txt.gz 2025-09-28 04:15:46,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:46,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:46,178 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-68d8b5f2a178a5bbc0ea6ac0/lava_callback.json.gz 2025-09-28 04:15:46,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:46,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f2a178a5bbc0ea6ac0 HTTP/1.1" 200 None 2025-09-28 04:15:46,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:46,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:46,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:46,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5f2a178a5bbc0ea6ac0 HTTP/1.1" 200 None 2025-09-28 04:15:46,288 - lava_callback - INFO - Completed processing callback for node 68d8b5f2a178a5bbc0ea6ac0 INFO: 172.21.0.1:43190 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:15:49,233 - lava_callback - INFO - Received callback for node 68d8b5dda178a5bbc0ea6a2d INFO: 172.21.0.1:43200 - "POST /node/68d8b5dda178a5bbc0ea6a2d HTTP/1.0" 202 Accepted 2025-09-28 04:15:49,239 - lava_callback - INFO - Starting async job submit for node 68d8b5dda178a5bbc0ea6a2d 2025-09-28 04:15:49,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:49,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dda178a5bbc0ea6a2d HTTP/1.1" 200 None 2025-09-28 04:15:49,277 - lava_callback - DEBUG - Processing node 68d8b5dda178a5bbc0ea6a2d: baseline-arm64 2025-09-28 04:15:49,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:49,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:49,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dda178a5bbc0ea6a2d/log.txt.gz 2025-09-28 04:15:49,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:49,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:49,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dda178a5bbc0ea6a2d/lava_callback.json.gz 2025-09-28 04:15:50,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:50,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dda178a5bbc0ea6a2d HTTP/1.1" 200 None 2025-09-28 04:15:50,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:50,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:50,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:50,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dda178a5bbc0ea6a2d HTTP/1.1" 200 None 2025-09-28 04:15:50,091 - lava_callback - INFO - Completed processing callback for node 68d8b5dda178a5bbc0ea6a2d 2025-09-28 04:15:51,753 - lava_callback - INFO - Received callback for node 68d8b5eba178a5bbc0ea6a93 INFO: 172.21.0.1:37044 - "POST /node/68d8b5eba178a5bbc0ea6a93 HTTP/1.0" 202 Accepted 2025-09-28 04:15:51,764 - lava_callback - INFO - Starting async job submit for node 68d8b5eba178a5bbc0ea6a93 2025-09-28 04:15:51,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:51,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eba178a5bbc0ea6a93 HTTP/1.1" 200 None 2025-09-28 04:15:51,813 - lava_callback - DEBUG - Processing node 68d8b5eba178a5bbc0ea6a93: baseline-arm64-broonie 2025-09-28 04:15:52,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:52,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:52,230 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eba178a5bbc0ea6a93/log.txt.gz 2025-09-28 04:15:52,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:52,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:52,399 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eba178a5bbc0ea6a93/lava_callback.json.gz 2025-09-28 04:15:52,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:52,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eba178a5bbc0ea6a93 HTTP/1.1" 200 None 2025-09-28 04:15:52,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:52,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:52,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:52,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eba178a5bbc0ea6a93 HTTP/1.1" 200 None 2025-09-28 04:15:52,487 - lava_callback - INFO - Completed processing callback for node 68d8b5eba178a5bbc0ea6a93 2025-09-28 04:15:53,102 - lava_callback - INFO - Received callback for node 68d8b5eda178a5bbc0ea6aa2 INFO: 172.21.0.1:37056 - "POST /node/68d8b5eda178a5bbc0ea6aa2 HTTP/1.0" 202 Accepted 2025-09-28 04:15:53,110 - lava_callback - INFO - Starting async job submit for node 68d8b5eda178a5bbc0ea6aa2 2025-09-28 04:15:53,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:53,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eda178a5bbc0ea6aa2 HTTP/1.1" 200 None 2025-09-28 04:15:53,149 - lava_callback - DEBUG - Processing node 68d8b5eda178a5bbc0ea6aa2: baseline-arm64-broonie 2025-09-28 04:15:53,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:53,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:53,644 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eda178a5bbc0ea6aa2/log.txt.gz 2025-09-28 04:15:53,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:15:53,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:15:53,803 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eda178a5bbc0ea6aa2/lava_callback.json.gz 2025-09-28 04:15:53,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:53,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eda178a5bbc0ea6aa2 HTTP/1.1" 200 None 2025-09-28 04:15:53,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:53,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:15:53,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:15:53,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eda178a5bbc0ea6aa2 HTTP/1.1" 200 None 2025-09-28 04:15:53,979 - lava_callback - INFO - Completed processing callback for node 68d8b5eda178a5bbc0ea6aa2 INFO: 172.21.0.1:37070 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:16:02,813 - lava_callback - INFO - Received callback for node 68d8b5e8a178a5bbc0ea6a7b INFO: 172.21.0.1:50274 - "POST /node/68d8b5e8a178a5bbc0ea6a7b HTTP/1.0" 202 Accepted 2025-09-28 04:16:02,819 - lava_callback - INFO - Starting async job submit for node 68d8b5e8a178a5bbc0ea6a7b 2025-09-28 04:16:02,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:02,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e8a178a5bbc0ea6a7b HTTP/1.1" 200 None 2025-09-28 04:16:02,853 - lava_callback - DEBUG - Processing node 68d8b5e8a178a5bbc0ea6a7b: baseline-arm64-broonie 2025-09-28 04:16:03,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:03,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:03,279 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e8a178a5bbc0ea6a7b/log.txt.gz 2025-09-28 04:16:03,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:16:03,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:03,460 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e8a178a5bbc0ea6a7b/lava_callback.json.gz 2025-09-28 04:16:03,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:03,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e8a178a5bbc0ea6a7b HTTP/1.1" 200 None 2025-09-28 04:16:03,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:03,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:16:03,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:03,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e8a178a5bbc0ea6a7b HTTP/1.1" 200 None 2025-09-28 04:16:03,560 - lava_callback - INFO - Completed processing callback for node 68d8b5e8a178a5bbc0ea6a7b INFO: 172.21.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50290 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:16:09,809 - lava_callback - INFO - Received callback for node 68d8b5e1a178a5bbc0ea6a51 INFO: 172.21.0.1:56814 - "POST /node/68d8b5e1a178a5bbc0ea6a51 HTTP/1.0" 202 Accepted 2025-09-28 04:16:09,815 - lava_callback - INFO - Starting async job submit for node 68d8b5e1a178a5bbc0ea6a51 2025-09-28 04:16:09,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:09,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e1a178a5bbc0ea6a51 HTTP/1.1" 200 None 2025-09-28 04:16:09,870 - lava_callback - DEBUG - Processing node 68d8b5e1a178a5bbc0ea6a51: baseline-arm64 2025-09-28 04:16:10,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:10,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:10,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e1a178a5bbc0ea6a51/log.txt.gz 2025-09-28 04:16:10,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:10,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:10,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e1a178a5bbc0ea6a51/lava_callback.json.gz 2025-09-28 04:16:10,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:10,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e1a178a5bbc0ea6a51 HTTP/1.1" 200 None 2025-09-28 04:16:10,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:10,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:16:10,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:10,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e1a178a5bbc0ea6a51 HTTP/1.1" 200 None 2025-09-28 04:16:10,590 - lava_callback - INFO - Completed processing callback for node 68d8b5e1a178a5bbc0ea6a51 INFO: 172.21.0.1:56824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53156 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:16:41,456 - lava_callback - INFO - Received callback for node 68d8b286a178a5bbc0ea6452 INFO: 172.21.0.1:57482 - "POST /node/68d8b286a178a5bbc0ea6452 HTTP/1.0" 202 Accepted 2025-09-28 04:16:41,468 - lava_callback - INFO - Starting async job submit for node 68d8b286a178a5bbc0ea6452 2025-09-28 04:16:41,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:41,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b286a178a5bbc0ea6452 HTTP/1.1" 200 None 2025-09-28 04:16:41,600 - lava_callback - DEBUG - Processing node 68d8b286a178a5bbc0ea6452: ltp-smoketest 2025-09-28 04:16:42,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:42,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:42,294 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8b286a178a5bbc0ea6452/log.txt.gz 2025-09-28 04:16:42,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:42,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:42,459 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8b286a178a5bbc0ea6452/lava_callback.json.gz 2025-09-28 04:16:42,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:42,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b286a178a5bbc0ea6452 HTTP/1.1" 200 None 2025-09-28 04:16:42,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:42,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae6a178a5bbc0ea5cb4 HTTP/1.1" 200 None 2025-09-28 04:16:42,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:42,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b286a178a5bbc0ea6452 HTTP/1.1" 200 None 2025-09-28 04:16:42,657 - lava_callback - INFO - Completed processing callback for node 68d8b286a178a5bbc0ea6452 2025-09-28 04:16:44,290 - lava_callback - INFO - Received callback for node 68d8b5eba178a5bbc0ea6a90 INFO: 172.21.0.1:57486 - "POST /node/68d8b5eba178a5bbc0ea6a90 HTTP/1.0" 202 Accepted 2025-09-28 04:16:44,296 - lava_callback - INFO - Starting async job submit for node 68d8b5eba178a5bbc0ea6a90 2025-09-28 04:16:44,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:44,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eba178a5bbc0ea6a90 HTTP/1.1" 200 None 2025-09-28 04:16:44,346 - lava_callback - DEBUG - Processing node 68d8b5eba178a5bbc0ea6a90: baseline-arm64-broonie 2025-09-28 04:16:44,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:44,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:44,899 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eba178a5bbc0ea6a90/log.txt.gz 2025-09-28 04:16:44,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:45,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:45,042 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eba178a5bbc0ea6a90/lava_callback.json.gz 2025-09-28 04:16:45,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:45,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eba178a5bbc0ea6a90 HTTP/1.1" 200 None 2025-09-28 04:16:45,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:45,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:16:45,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:45,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eba178a5bbc0ea6a90 HTTP/1.1" 200 None 2025-09-28 04:16:45,159 - lava_callback - INFO - Completed processing callback for node 68d8b5eba178a5bbc0ea6a90 INFO: 172.21.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50198 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:16:57,737 - lava_callback - INFO - Received callback for node 68d8b5e0a178a5bbc0ea6a48 INFO: 172.21.0.1:50202 - "POST /node/68d8b5e0a178a5bbc0ea6a48 HTTP/1.0" 202 Accepted 2025-09-28 04:16:57,743 - lava_callback - INFO - Starting async job submit for node 68d8b5e0a178a5bbc0ea6a48 2025-09-28 04:16:57,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:57,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e0a178a5bbc0ea6a48 HTTP/1.1" 200 None 2025-09-28 04:16:57,776 - lava_callback - DEBUG - Processing node 68d8b5e0a178a5bbc0ea6a48: baseline-arm64 2025-09-28 04:16:58,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:58,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:58,313 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e0a178a5bbc0ea6a48/log.txt.gz 2025-09-28 04:16:58,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:16:58,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:16:58,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e0a178a5bbc0ea6a48/lava_callback.json.gz 2025-09-28 04:16:58,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:58,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e0a178a5bbc0ea6a48 HTTP/1.1" 200 None 2025-09-28 04:16:58,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:58,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:16:58,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:16:58,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e0a178a5bbc0ea6a48 HTTP/1.1" 200 None 2025-09-28 04:16:58,579 - lava_callback - INFO - Completed processing callback for node 68d8b5e0a178a5bbc0ea6a48 INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43038 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:17:35,187 - lava_callback - INFO - Received callback for node 68d8b5e3a178a5bbc0ea6a57 INFO: 172.21.0.1:51638 - "POST /node/68d8b5e3a178a5bbc0ea6a57 HTTP/1.0" 202 Accepted 2025-09-28 04:17:35,193 - lava_callback - INFO - Starting async job submit for node 68d8b5e3a178a5bbc0ea6a57 2025-09-28 04:17:35,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:35,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e3a178a5bbc0ea6a57 HTTP/1.1" 200 None 2025-09-28 04:17:35,249 - lava_callback - DEBUG - Processing node 68d8b5e3a178a5bbc0ea6a57: baseline-arm64-baylibre 2025-09-28 04:17:35,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:35,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:35,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e3a178a5bbc0ea6a57/log.txt.gz 2025-09-28 04:17:35,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:35,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:35,867 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e3a178a5bbc0ea6a57/lava_callback.json.gz 2025-09-28 04:17:35,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:35,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e3a178a5bbc0ea6a57 HTTP/1.1" 200 None 2025-09-28 04:17:35,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:35,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:17:35,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:35,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e3a178a5bbc0ea6a57 HTTP/1.1" 200 None 2025-09-28 04:17:35,960 - lava_callback - INFO - Completed processing callback for node 68d8b5e3a178a5bbc0ea6a57 INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:17:38,870 - lava_callback - INFO - Received callback for node 68d8b658a178a5bbc0ea6b10 INFO: 172.21.0.1:51666 - "POST /node/68d8b658a178a5bbc0ea6b10 HTTP/1.0" 202 Accepted 2025-09-28 04:17:38,878 - lava_callback - INFO - Starting async job submit for node 68d8b658a178a5bbc0ea6b10 2025-09-28 04:17:38,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:38,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b658a178a5bbc0ea6b10 HTTP/1.1" 200 None 2025-09-28 04:17:38,927 - lava_callback - DEBUG - Processing node 68d8b658a178a5bbc0ea6b10: baseline-arm64-kontron 2025-09-28 04:17:39,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:39,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:39,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b658a178a5bbc0ea6b10/log.txt.gz 2025-09-28 04:17:39,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:39,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:39,535 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b658a178a5bbc0ea6b10/lava_callback.json.gz 2025-09-28 04:17:39,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:39,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b658a178a5bbc0ea6b10 HTTP/1.1" 200 None 2025-09-28 04:17:39,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:39,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:17:39,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:39,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b658a178a5bbc0ea6b10 HTTP/1.1" 200 None 2025-09-28 04:17:39,638 - lava_callback - INFO - Completed processing callback for node 68d8b658a178a5bbc0ea6b10 2025-09-28 04:17:44,720 - lava_callback - INFO - Received callback for node 68d8b5e4a178a5bbc0ea6a5d INFO: 172.21.0.1:39358 - "POST /node/68d8b5e4a178a5bbc0ea6a5d HTTP/1.0" 202 Accepted 2025-09-28 04:17:44,727 - lava_callback - INFO - Starting async job submit for node 68d8b5e4a178a5bbc0ea6a5d 2025-09-28 04:17:44,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:44,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e4a178a5bbc0ea6a5d HTTP/1.1" 200 None 2025-09-28 04:17:44,763 - lava_callback - DEBUG - Processing node 68d8b5e4a178a5bbc0ea6a5d: baseline-arm64-baylibre 2025-09-28 04:17:45,028 - lava_callback - INFO - Received callback for node 68d8b5eaa178a5bbc0ea6a8a INFO: 172.21.0.1:39366 - "POST /node/68d8b5eaa178a5bbc0ea6a8a HTTP/1.0" 202 Accepted 2025-09-28 04:17:45,050 - lava_callback - INFO - Starting async job submit for node 68d8b5eaa178a5bbc0ea6a8a 2025-09-28 04:17:45,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:45,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eaa178a5bbc0ea6a8a HTTP/1.1" 200 None 2025-09-28 04:17:45,196 - lava_callback - DEBUG - Processing node 68d8b5eaa178a5bbc0ea6a8a: baseline-arm64-broonie 2025-09-28 04:17:45,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:45,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:45,462 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e4a178a5bbc0ea6a5d/log.txt.gz 2025-09-28 04:17:45,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:45,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:45,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:45,749 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e4a178a5bbc0ea6a5d/lava_callback.json.gz 2025-09-28 04:17:45,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:45,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:45,788 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eaa178a5bbc0ea6a8a/log.txt.gz 2025-09-28 04:17:45,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:17:45,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e4a178a5bbc0ea6a5d HTTP/1.1" 200 None 2025-09-28 04:17:45,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:45,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:17:45,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:45,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e4a178a5bbc0ea6a5d HTTP/1.1" 200 None 2025-09-28 04:17:45,873 - lava_callback - INFO - Completed processing callback for node 68d8b5e4a178a5bbc0ea6a5d 2025-09-28 04:17:46,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:17:46,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5eaa178a5bbc0ea6a8a/lava_callback.json.gz 2025-09-28 04:17:46,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:46,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5eaa178a5bbc0ea6a8a HTTP/1.1" 200 None 2025-09-28 04:17:46,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:46,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:17:46,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:17:46,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5eaa178a5bbc0ea6a8a HTTP/1.1" 200 None 2025-09-28 04:17:46,120 - lava_callback - INFO - Completed processing callback for node 68d8b5eaa178a5bbc0ea6a8a INFO: 172.21.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60678 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:18:01,578 - lava_callback - INFO - Received callback for node 68d8abf6a178a5bbc0ea5d4e INFO: 172.21.0.1:37792 - "POST /node/68d8abf6a178a5bbc0ea5d4e HTTP/1.0" 202 Accepted 2025-09-28 04:18:01,624 - lava_callback - INFO - Starting async job submit for node 68d8abf6a178a5bbc0ea5d4e 2025-09-28 04:18:01,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:01,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8abf6a178a5bbc0ea5d4e HTTP/1.1" 200 None 2025-09-28 04:18:01,664 - lava_callback - DEBUG - Processing node 68d8abf6a178a5bbc0ea5d4e: baseline-x86 2025-09-28 04:18:01,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:01,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:01,918 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8abf6a178a5bbc0ea5d4e/log.txt.gz 2025-09-28 04:18:01,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:02,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:02,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8abf6a178a5bbc0ea5d4e/lava_callback.json.gz 2025-09-28 04:18:02,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:02,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8abf6a178a5bbc0ea5d4e HTTP/1.1" 200 None 2025-09-28 04:18:02,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:02,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f5b7aff7facd9a8d14f HTTP/1.1" 200 None 2025-09-28 04:18:02,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:02,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8abf6a178a5bbc0ea5d4e HTTP/1.1" 200 None 2025-09-28 04:18:02,246 - lava_callback - INFO - Completed processing callback for node 68d8abf6a178a5bbc0ea5d4e INFO: 172.21.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37830 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:18:14,390 - lava_callback - INFO - Received callback for node 68d8b5e6a178a5bbc0ea6a72 INFO: 172.21.0.1:43398 - "POST /node/68d8b5e6a178a5bbc0ea6a72 HTTP/1.0" 202 Accepted 2025-09-28 04:18:14,396 - lava_callback - INFO - Starting async job submit for node 68d8b5e6a178a5bbc0ea6a72 2025-09-28 04:18:14,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:14,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e6a178a5bbc0ea6a72 HTTP/1.1" 200 None 2025-09-28 04:18:14,442 - lava_callback - DEBUG - Processing node 68d8b5e6a178a5bbc0ea6a72: baseline-arm64-baylibre 2025-09-28 04:18:14,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:15,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:15,049 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e6a178a5bbc0ea6a72/log.txt.gz 2025-09-28 04:18:15,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:15,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:15,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e6a178a5bbc0ea6a72/lava_callback.json.gz INFO: 172.21.0.1:43402 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 04:18:15,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:15,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e6a178a5bbc0ea6a72 HTTP/1.1" 200 None 2025-09-28 04:18:15,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:15,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:18:15,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:15,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e6a178a5bbc0ea6a72 HTTP/1.1" 200 None 2025-09-28 04:18:15,283 - lava_callback - INFO - Completed processing callback for node 68d8b5e6a178a5bbc0ea6a72 INFO: 172.21.0.1:43410 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:18:17,740 - lava_callback - INFO - Received callback for node 68d8b5e3a178a5bbc0ea6a5a INFO: 172.21.0.1:43422 - "POST /node/68d8b5e3a178a5bbc0ea6a5a HTTP/1.0" 202 Accepted 2025-09-28 04:18:17,746 - lava_callback - INFO - Starting async job submit for node 68d8b5e3a178a5bbc0ea6a5a 2025-09-28 04:18:17,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:17,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e3a178a5bbc0ea6a5a HTTP/1.1" 200 None 2025-09-28 04:18:17,780 - lava_callback - DEBUG - Processing node 68d8b5e3a178a5bbc0ea6a5a: baseline-arm64-baylibre 2025-09-28 04:18:18,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:18,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:18,167 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e3a178a5bbc0ea6a5a/log.txt.gz 2025-09-28 04:18:18,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:18,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:18,302 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e3a178a5bbc0ea6a5a/lava_callback.json.gz 2025-09-28 04:18:18,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:18,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e3a178a5bbc0ea6a5a HTTP/1.1" 200 None 2025-09-28 04:18:18,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:18,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:18:18,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:18,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e3a178a5bbc0ea6a5a HTTP/1.1" 200 None 2025-09-28 04:18:18,385 - lava_callback - INFO - Completed processing callback for node 68d8b5e3a178a5bbc0ea6a5a 2025-09-28 04:18:20,089 - lava_callback - INFO - Received callback for node 68d8b5e4a178a5bbc0ea6a63 INFO: 172.21.0.1:58632 - "POST /node/68d8b5e4a178a5bbc0ea6a63 HTTP/1.0" 202 Accepted 2025-09-28 04:18:20,095 - lava_callback - INFO - Starting async job submit for node 68d8b5e4a178a5bbc0ea6a63 2025-09-28 04:18:20,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:20,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e4a178a5bbc0ea6a63 HTTP/1.1" 200 None 2025-09-28 04:18:20,132 - lava_callback - DEBUG - Processing node 68d8b5e4a178a5bbc0ea6a63: baseline-arm64-baylibre 2025-09-28 04:18:20,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:20,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:20,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e4a178a5bbc0ea6a63/log.txt.gz 2025-09-28 04:18:20,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:20,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:20,636 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e4a178a5bbc0ea6a63/lava_callback.json.gz 2025-09-28 04:18:20,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:20,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e4a178a5bbc0ea6a63 HTTP/1.1" 200 None 2025-09-28 04:18:20,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:20,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:18:20,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:20,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e4a178a5bbc0ea6a63 HTTP/1.1" 200 None 2025-09-28 04:18:20,718 - lava_callback - INFO - Completed processing callback for node 68d8b5e4a178a5bbc0ea6a63 INFO: 172.21.0.1:58648 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:18:31,073 - lava_callback - INFO - Received callback for node 68d8b5e5a178a5bbc0ea6a66 INFO: 172.21.0.1:58696 - "POST /node/68d8b5e5a178a5bbc0ea6a66 HTTP/1.0" 202 Accepted 2025-09-28 04:18:31,079 - lava_callback - INFO - Starting async job submit for node 68d8b5e5a178a5bbc0ea6a66 2025-09-28 04:18:31,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:31,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e5a178a5bbc0ea6a66 HTTP/1.1" 200 None 2025-09-28 04:18:31,113 - lava_callback - DEBUG - Processing node 68d8b5e5a178a5bbc0ea6a66: baseline-arm64-baylibre 2025-09-28 04:18:31,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:31,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:31,501 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e5a178a5bbc0ea6a66/log.txt.gz 2025-09-28 04:18:31,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:18:31,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:18:31,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e5a178a5bbc0ea6a66/lava_callback.json.gz 2025-09-28 04:18:31,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:31,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e5a178a5bbc0ea6a66 HTTP/1.1" 200 None 2025-09-28 04:18:31,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:31,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:18:31,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:18:31,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e5a178a5bbc0ea6a66 HTTP/1.1" 200 None 2025-09-28 04:18:31,692 - lava_callback - INFO - Completed processing callback for node 68d8b5e5a178a5bbc0ea6a66 INFO: 172.21.0.1:58706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52204 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:19:39,030 - lava_callback - INFO - Received callback for node 68d8b104a178a5bbc0ea630a INFO: 172.21.0.1:52218 - "POST /node/68d8b104a178a5bbc0ea630a HTTP/1.0" 202 Accepted 2025-09-28 04:19:39,037 - lava_callback - INFO - Starting async job submit for node 68d8b104a178a5bbc0ea630a 2025-09-28 04:19:39,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:39,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b104a178a5bbc0ea630a HTTP/1.1" 200 None 2025-09-28 04:19:39,086 - lava_callback - DEBUG - Processing node 68d8b104a178a5bbc0ea630a: job-gcc-12-arm-612-cip 2025-09-28 04:19:39,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:19:39,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:19:39,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/job-gcc-12-arm-612-cip-68d8b104a178a5bbc0ea630a/log.txt.gz 2025-09-28 04:19:39,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:19:39,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:19:39,648 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/job-gcc-12-arm-612-cip-68d8b104a178a5bbc0ea630a/lava_callback.json.gz 2025-09-28 04:19:39,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:39,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b104a178a5bbc0ea630a HTTP/1.1" 200 None 2025-09-28 04:19:39,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:39,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f247aff7facd9a8d0e7 HTTP/1.1" 200 None 2025-09-28 04:19:39,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:39,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b104a178a5bbc0ea630a HTTP/1.1" 200 None 2025-09-28 04:19:39,775 - lava_callback - INFO - Completed processing callback for node 68d8b104a178a5bbc0ea630a INFO: 172.21.0.1:56228 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:19:48,272 - lava_callback - INFO - Received callback for node 68d8b5e5a178a5bbc0ea6a6f INFO: 172.21.0.1:56234 - "POST /node/68d8b5e5a178a5bbc0ea6a6f HTTP/1.0" 202 Accepted 2025-09-28 04:19:48,278 - lava_callback - INFO - Starting async job submit for node 68d8b5e5a178a5bbc0ea6a6f 2025-09-28 04:19:48,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:48,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e5a178a5bbc0ea6a6f HTTP/1.1" 200 None 2025-09-28 04:19:48,342 - lava_callback - DEBUG - Processing node 68d8b5e5a178a5bbc0ea6a6f: baseline-arm64-baylibre 2025-09-28 04:19:48,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:19:48,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:19:48,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e5a178a5bbc0ea6a6f/log.txt.gz 2025-09-28 04:19:48,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:19:49,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:19:49,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-baylibre-68d8b5e5a178a5bbc0ea6a6f/lava_callback.json.gz 2025-09-28 04:19:49,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:49,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e5a178a5bbc0ea6a6f HTTP/1.1" 200 None 2025-09-28 04:19:49,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:49,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:19:49,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:19:49,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e5a178a5bbc0ea6a6f HTTP/1.1" 200 None 2025-09-28 04:19:49,127 - lava_callback - INFO - Completed processing callback for node 68d8b5e5a178a5bbc0ea6a6f INFO: 172.21.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:20:18,966 - lava_callback - INFO - Received callback for node 68d8b6e4a178a5bbc0ea6b6c INFO: 172.21.0.1:42084 - "POST /node/68d8b6e4a178a5bbc0ea6b6c HTTP/1.0" 202 Accepted 2025-09-28 04:20:18,972 - lava_callback - INFO - Starting async job submit for node 68d8b6e4a178a5bbc0ea6b6c 2025-09-28 04:20:19,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:19,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b6e4a178a5bbc0ea6b6c HTTP/1.1" 200 None 2025-09-28 04:20:19,018 - lava_callback - DEBUG - Processing node 68d8b6e4a178a5bbc0ea6b6c: baseline-arm64-kontron 2025-09-28 04:20:19,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:20:19,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:20:19,370 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b6e4a178a5bbc0ea6b6c/log.txt.gz 2025-09-28 04:20:19,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:20:19,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:20:19,518 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b6e4a178a5bbc0ea6b6c/lava_callback.json.gz 2025-09-28 04:20:19,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:19,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b6e4a178a5bbc0ea6b6c HTTP/1.1" 200 None 2025-09-28 04:20:19,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:19,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:20:19,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:19,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b6e4a178a5bbc0ea6b6c HTTP/1.1" 200 None 2025-09-28 04:20:19,611 - lava_callback - INFO - Completed processing callback for node 68d8b6e4a178a5bbc0ea6b6c INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53818 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:20:45,291 - lava_callback - INFO - Received callback for node 68d8b1bba178a5bbc0ea6341 INFO: 172.21.0.1:46852 - "POST /node/68d8b1bba178a5bbc0ea6341 HTTP/1.0" 202 Accepted 2025-09-28 04:20:45,297 - lava_callback - INFO - Starting async job submit for node 68d8b1bba178a5bbc0ea6341 2025-09-28 04:20:45,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:45,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bba178a5bbc0ea6341 HTTP/1.1" 200 None 2025-09-28 04:20:45,329 - lava_callback - DEBUG - Processing node 68d8b1bba178a5bbc0ea6341: baseline-arm 2025-09-28 04:20:45,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:20:45,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:20:45,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bba178a5bbc0ea6341/log.txt.gz 2025-09-28 04:20:45,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:20:45,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:20:45,848 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bba178a5bbc0ea6341/lava_callback.json.gz 2025-09-28 04:20:45,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:45,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bba178a5bbc0ea6341 HTTP/1.1" 200 None 2025-09-28 04:20:45,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:45,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:20:45,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:20:45,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bba178a5bbc0ea6341 HTTP/1.1" 200 None 2025-09-28 04:20:45,926 - lava_callback - INFO - Completed processing callback for node 68d8b1bba178a5bbc0ea6341 INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42982 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:21:01,667 - lava_callback - INFO - Received callback for node 68d7fb148dd6b39fd1aeb807 INFO: 172.21.0.1:44584 - "POST /node/68d7fb148dd6b39fd1aeb807 HTTP/1.0" 202 Accepted 2025-09-28 04:21:01,674 - lava_callback - INFO - Starting async job submit for node 68d7fb148dd6b39fd1aeb807 2025-09-28 04:21:01,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:01,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fb148dd6b39fd1aeb807 HTTP/1.1" 200 None 2025-09-28 04:21:01,727 - lava_callback - DEBUG - Processing node 68d7fb148dd6b39fd1aeb807: baseline-arm64 2025-09-28 04:21:02,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:21:02,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:21:02,740 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fb148dd6b39fd1aeb807/log.txt.gz 2025-09-28 04:21:02,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:21:02,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:21:02,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fb148dd6b39fd1aeb807/lava_callback.json.gz DEBUG: Setup failed for 68d7fb148dd6b39fd1aeb807. Transitting job node result: incomplete -> fail 2025-09-28 04:21:03,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:03,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fb148dd6b39fd1aeb807 HTTP/1.1" 200 None 2025-09-28 04:21:03,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:03,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d79eda8dd6b39fd1aea2de HTTP/1.1" 200 None 2025-09-28 04:21:03,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:03,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fb148dd6b39fd1aeb807 HTTP/1.1" 200 None 2025-09-28 04:21:03,123 - lava_callback - INFO - Completed processing callback for node 68d7fb148dd6b39fd1aeb807 INFO: 172.21.0.1:44600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45976 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:21:21,688 - lava_callback - INFO - Received callback for node 68d8b1bca178a5bbc0ea634d INFO: 172.21.0.1:44812 - "POST /node/68d8b1bca178a5bbc0ea634d HTTP/1.0" 202 Accepted 2025-09-28 04:21:21,694 - lava_callback - INFO - Starting async job submit for node 68d8b1bca178a5bbc0ea634d 2025-09-28 04:21:21,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:21,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bca178a5bbc0ea634d HTTP/1.1" 200 None 2025-09-28 04:21:21,728 - lava_callback - DEBUG - Processing node 68d8b1bca178a5bbc0ea634d: baseline-arm 2025-09-28 04:21:21,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:21:22,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:21:22,059 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bca178a5bbc0ea634d/log.txt.gz 2025-09-28 04:21:22,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:21:22,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:21:22,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b1bca178a5bbc0ea634d/lava_callback.json.gz 2025-09-28 04:21:22,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:22,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1bca178a5bbc0ea634d HTTP/1.1" 200 None 2025-09-28 04:21:22,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:22,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:21:22,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:21:22,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1bca178a5bbc0ea634d HTTP/1.1" 200 None 2025-09-28 04:21:22,251 - lava_callback - INFO - Completed processing callback for node 68d8b1bca178a5bbc0ea634d INFO: 172.21.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:22:59,032 - lava_callback - INFO - Received callback for node 68d8b784a178a5bbc0ea6b96 INFO: 172.21.0.1:59192 - "POST /node/68d8b784a178a5bbc0ea6b96 HTTP/1.0" 202 Accepted 2025-09-28 04:22:59,038 - lava_callback - INFO - Starting async job submit for node 68d8b784a178a5bbc0ea6b96 2025-09-28 04:22:59,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:22:59,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b784a178a5bbc0ea6b96 HTTP/1.1" 200 None 2025-09-28 04:22:59,106 - lava_callback - DEBUG - Processing node 68d8b784a178a5bbc0ea6b96: baseline-arm64-kontron 2025-09-28 04:22:59,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:22:59,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:22:59,473 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b784a178a5bbc0ea6b96/log.txt.gz 2025-09-28 04:22:59,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:22:59,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:22:59,605 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-68d8b784a178a5bbc0ea6b96/lava_callback.json.gz 2025-09-28 04:22:59,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:22:59,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b784a178a5bbc0ea6b96 HTTP/1.1" 200 None 2025-09-28 04:22:59,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:22:59,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:22:59,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:22:59,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b784a178a5bbc0ea6b96 HTTP/1.1" 200 None 2025-09-28 04:22:59,700 - lava_callback - INFO - Completed processing callback for node 68d8b784a178a5bbc0ea6b96 INFO: 172.21.0.1:56788 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:23:06,017 - lava_callback - INFO - Received callback for node 68d8b1d2a178a5bbc0ea63da INFO: 172.21.0.1:56792 - "POST /node/68d8b1d2a178a5bbc0ea63da HTTP/1.0" 202 Accepted 2025-09-28 04:23:06,024 - lava_callback - INFO - Starting async job submit for node 68d8b1d2a178a5bbc0ea63da 2025-09-28 04:23:06,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:23:06,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1d2a178a5bbc0ea63da HTTP/1.1" 200 None 2025-09-28 04:23:06,070 - lava_callback - DEBUG - Processing node 68d8b1d2a178a5bbc0ea63da: ltp-dio 2025-09-28 04:23:06,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:23:06,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:23:06,609 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-68d8b1d2a178a5bbc0ea63da/log.txt.gz 2025-09-28 04:23:06,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:23:06,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:23:06,752 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-68d8b1d2a178a5bbc0ea63da/lava_callback.json.gz 2025-09-28 04:23:06,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:23:06,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b1d2a178a5bbc0ea63da HTTP/1.1" 200 None 2025-09-28 04:23:06,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:23:06,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:23:06,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:23:06,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b1d2a178a5bbc0ea63da HTTP/1.1" 200 None 2025-09-28 04:23:06,921 - lava_callback - INFO - Completed processing callback for node 68d8b1d2a178a5bbc0ea63da INFO: 172.21.0.1:56802 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:23:12,179 - lava_callback - INFO - Received callback for node 68d6179630c58dac7d941fd6 INFO: 172.21.0.1:54044 - "POST /node/68d6179630c58dac7d941fd6 HTTP/1.0" 202 Accepted 2025-09-28 04:23:12,186 - lava_callback - INFO - Starting async job submit for node 68d6179630c58dac7d941fd6 2025-09-28 04:23:12,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:23:12,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179630c58dac7d941fd6 HTTP/1.1" 200 None 2025-09-28 04:23:12,237 - lava_callback - DEBUG - Processing node 68d6179630c58dac7d941fd6: ltp-pty 2025-09-28 04:23:12,600 - lava_callback - ERROR - Error processing callback for node 68d6179630c58dac7d941fd6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:24:03,065 - lava_callback - INFO - Received callback for node 68d8aa8ca178a5bbc0ea5c8f INFO: 172.21.0.1:49074 - "POST /node/68d8aa8ca178a5bbc0ea5c8f HTTP/1.0" 202 Accepted 2025-09-28 04:24:03,071 - lava_callback - INFO - Starting async job submit for node 68d8aa8ca178a5bbc0ea5c8f 2025-09-28 04:24:03,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:24:03,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa8ca178a5bbc0ea5c8f HTTP/1.1" 200 None 2025-09-28 04:24:03,115 - lava_callback - DEBUG - Processing node 68d8aa8ca178a5bbc0ea5c8f: baseline-arm 2025-09-28 04:24:03,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:24:03,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:24:03,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8aa8ca178a5bbc0ea5c8f/log.txt.gz 2025-09-28 04:24:03,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:24:03,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:24:03,680 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8aa8ca178a5bbc0ea5c8f/lava_callback.json.gz DEBUG: Setup failed for 68d8aa8ca178a5bbc0ea5c8f. Transitting job node result: incomplete -> fail 2025-09-28 04:24:03,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:24:03,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aa8ca178a5bbc0ea5c8f HTTP/1.1" 200 None 2025-09-28 04:24:03,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:24:03,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:24:03,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:24:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8aa8ca178a5bbc0ea5c8f HTTP/1.1" 200 None 2025-09-28 04:24:03,788 - lava_callback - INFO - Completed processing callback for node 68d8aa8ca178a5bbc0ea5c8f INFO: 172.21.0.1:49082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39712 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:25:51,476 - lava_callback - INFO - Received callback for node 68d7fb158dd6b39fd1aeb80e INFO: 172.21.0.1:52146 - "POST /node/68d7fb158dd6b39fd1aeb80e HTTP/1.0" 202 Accepted 2025-09-28 04:25:51,483 - lava_callback - INFO - Starting async job submit for node 68d7fb158dd6b39fd1aeb80e 2025-09-28 04:25:51,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:25:51,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fb158dd6b39fd1aeb80e HTTP/1.1" 200 None 2025-09-28 04:25:51,540 - lava_callback - DEBUG - Processing node 68d7fb158dd6b39fd1aeb80e: baseline-arm64 2025-09-28 04:25:52,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:25:52,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:25:52,592 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fb158dd6b39fd1aeb80e/log.txt.gz 2025-09-28 04:25:52,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:25:52,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:25:52,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fb158dd6b39fd1aeb80e/lava_callback.json.gz DEBUG: Setup failed for 68d7fb158dd6b39fd1aeb80e. Transitting job node result: incomplete -> fail 2025-09-28 04:25:52,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:25:52,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fb158dd6b39fd1aeb80e HTTP/1.1" 200 None 2025-09-28 04:25:52,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:25:52,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d79eda8dd6b39fd1aea2de HTTP/1.1" 200 None 2025-09-28 04:25:52,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:25:52,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fb158dd6b39fd1aeb80e HTTP/1.1" 200 None 2025-09-28 04:25:52,913 - lava_callback - INFO - Completed processing callback for node 68d7fb158dd6b39fd1aeb80e INFO: 172.21.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47712 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:26:11,558 - lava_callback - INFO - Received callback for node 68d7fdaa8dd6b39fd1aeb852 INFO: 172.21.0.1:44996 - "POST /node/68d7fdaa8dd6b39fd1aeb852 HTTP/1.0" 202 Accepted 2025-09-28 04:26:11,569 - lava_callback - INFO - Starting async job submit for node 68d7fdaa8dd6b39fd1aeb852 2025-09-28 04:26:11,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:26:11,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdaa8dd6b39fd1aeb852 HTTP/1.1" 200 None 2025-09-28 04:26:11,635 - lava_callback - DEBUG - Processing node 68d7fdaa8dd6b39fd1aeb852: baseline-arm64 2025-09-28 04:26:12,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:26:13,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:26:13,040 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdaa8dd6b39fd1aeb852/log.txt.gz 2025-09-28 04:26:13,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:26:13,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:26:13,227 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdaa8dd6b39fd1aeb852/lava_callback.json.gz DEBUG: Setup failed for 68d7fdaa8dd6b39fd1aeb852. Transitting job node result: incomplete -> fail 2025-09-28 04:26:13,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:26:13,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdaa8dd6b39fd1aeb852 HTTP/1.1" 200 None 2025-09-28 04:26:13,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:26:13,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-28 04:26:13,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:26:13,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fdaa8dd6b39fd1aeb852 HTTP/1.1" 200 None 2025-09-28 04:26:13,378 - lava_callback - INFO - Completed processing callback for node 68d7fdaa8dd6b39fd1aeb852 INFO: 172.21.0.1:44998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:28:08,664 - lava_callback - INFO - Received callback for node 68d8b4dfa178a5bbc0ea6a14 INFO: 172.21.0.1:56734 - "POST /node/68d8b4dfa178a5bbc0ea6a14 HTTP/1.0" 202 Accepted 2025-09-28 04:28:08,672 - lava_callback - INFO - Starting async job submit for node 68d8b4dfa178a5bbc0ea6a14 2025-09-28 04:28:08,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:08,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b4dfa178a5bbc0ea6a14 HTTP/1.1" 200 None 2025-09-28 04:28:08,712 - lava_callback - DEBUG - Processing node 68d8b4dfa178a5bbc0ea6a14: ltp-syscalls-ipc INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:28:09,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:28:09,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:28:09,668 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8b4dfa178a5bbc0ea6a14/log.txt.gz 2025-09-28 04:28:09,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:28:09,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:28:09,844 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8b4dfa178a5bbc0ea6a14/lava_callback.json.gz 2025-09-28 04:28:09,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:09,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b4dfa178a5bbc0ea6a14 HTTP/1.1" 200 None 2025-09-28 04:28:09,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:09,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:28:09,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:09,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b4dfa178a5bbc0ea6a14 HTTP/1.1" 200 None 2025-09-28 04:28:09,969 - lava_callback - INFO - Completed processing callback for node 68d8b4dfa178a5bbc0ea6a14 INFO: 172.21.0.1:34556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:28:30,210 - lava_callback - INFO - Received callback for node 68d8b5dfa178a5bbc0ea6a3f INFO: 172.21.0.1:33216 - "POST /node/68d8b5dfa178a5bbc0ea6a3f HTTP/1.0" 202 Accepted 2025-09-28 04:28:30,217 - lava_callback - INFO - Starting async job submit for node 68d8b5dfa178a5bbc0ea6a3f 2025-09-28 04:28:30,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:30,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dfa178a5bbc0ea6a3f HTTP/1.1" 200 None 2025-09-28 04:28:30,285 - lava_callback - DEBUG - Processing node 68d8b5dfa178a5bbc0ea6a3f: baseline-arm64 2025-09-28 04:28:31,150 - lava_callback - INFO - Received callback for node 68d8b5f6a178a5bbc0ea6ad2 INFO: 172.21.0.1:33230 - "POST /node/68d8b5f6a178a5bbc0ea6ad2 HTTP/1.0" 202 Accepted 2025-09-28 04:28:31,182 - lava_callback - INFO - Starting async job submit for node 68d8b5f6a178a5bbc0ea6ad2 2025-09-28 04:28:31,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:31,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:28:31,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f6a178a5bbc0ea6ad2 HTTP/1.1" 200 None 2025-09-28 04:28:31,277 - lava_callback - DEBUG - Processing node 68d8b5f6a178a5bbc0ea6ad2: baseline-arm64-pengutronix 2025-09-28 04:28:31,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:28:31,441 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dfa178a5bbc0ea6a3f/log.txt.gz 2025-09-28 04:28:31,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:28:31,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:28:31,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:28:31,624 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d8b5f6a178a5bbc0ea6ad2/log.txt.gz 2025-09-28 04:28:31,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:28:31,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:28:31,643 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dfa178a5bbc0ea6a3f/lava_callback.json.gz DEBUG: Setup failed for 68d8b5dfa178a5bbc0ea6a3f. Transitting job node result: incomplete -> fail 2025-09-28 04:28:31,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:31,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dfa178a5bbc0ea6a3f HTTP/1.1" 200 None 2025-09-28 04:28:31,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:31,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:28:31,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:31,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dfa178a5bbc0ea6a3f HTTP/1.1" 200 None 2025-09-28 04:28:31,778 - lava_callback - INFO - Completed processing callback for node 68d8b5dfa178a5bbc0ea6a3f 2025-09-28 04:28:31,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:28:31,854 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-68d8b5f6a178a5bbc0ea6ad2/lava_callback.json.gz 2025-09-28 04:28:31,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5f6a178a5bbc0ea6ad2 HTTP/1.1" 200 None 2025-09-28 04:28:31,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:31,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:28:31,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:28:31,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5f6a178a5bbc0ea6ad2 HTTP/1.1" 200 None 2025-09-28 04:28:31,971 - lava_callback - INFO - Completed processing callback for node 68d8b5f6a178a5bbc0ea6ad2 INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47058 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:31:32,510 - lava_callback - INFO - Received callback for node 68d6179a30c58dac7d941fde INFO: 172.21.0.1:52048 - "POST /node/68d6179a30c58dac7d941fde HTTP/1.0" 202 Accepted 2025-09-28 04:31:32,517 - lava_callback - INFO - Starting async job submit for node 68d6179a30c58dac7d941fde 2025-09-28 04:31:32,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:31:32,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179a30c58dac7d941fde HTTP/1.1" 200 None 2025-09-28 04:31:32,564 - lava_callback - DEBUG - Processing node 68d6179a30c58dac7d941fde: ltp-smoketest 2025-09-28 04:31:32,962 - lava_callback - ERROR - Error processing callback for node 68d6179a30c58dac7d941fde: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60410 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:32:50,747 - lava_callback - INFO - Received callback for node 68d6175f30c58dac7d941f5b INFO: 172.21.0.1:35956 - "POST /node/68d6175f30c58dac7d941f5b HTTP/1.0" 202 Accepted 2025-09-28 04:32:50,752 - lava_callback - INFO - Starting async job submit for node 68d6175f30c58dac7d941f5b 2025-09-28 04:32:50,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:32:50,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6175f30c58dac7d941f5b HTTP/1.1" 200 None 2025-09-28 04:32:50,767 - lava_callback - DEBUG - Processing node 68d6175f30c58dac7d941f5b: baseline-arm 2025-09-28 04:32:50,767 - lava_callback - ERROR - Error processing callback for node 68d6175f30c58dac7d941f5b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:32:51,396 - lava_callback - INFO - Received callback for node 68d6176130c58dac7d941f5e INFO: 172.21.0.1:35966 - "POST /node/68d6176130c58dac7d941f5e HTTP/1.0" 202 Accepted 2025-09-28 04:32:51,401 - lava_callback - INFO - Starting async job submit for node 68d6176130c58dac7d941f5e 2025-09-28 04:32:51,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:32:51,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6176130c58dac7d941f5e HTTP/1.1" 200 None 2025-09-28 04:32:51,416 - lava_callback - DEBUG - Processing node 68d6176130c58dac7d941f5e: baseline-arm 2025-09-28 04:32:51,416 - lava_callback - ERROR - Error processing callback for node 68d6176130c58dac7d941f5e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:32:52,030 - lava_callback - INFO - Received callback for node 68d6176130c58dac7d941f61 INFO: 172.21.0.1:35970 - "POST /node/68d6176130c58dac7d941f61 HTTP/1.0" 202 Accepted 2025-09-28 04:32:52,036 - lava_callback - INFO - Starting async job submit for node 68d6176130c58dac7d941f61 2025-09-28 04:32:52,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:32:52,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6176130c58dac7d941f61 HTTP/1.1" 200 None 2025-09-28 04:32:52,055 - lava_callback - DEBUG - Processing node 68d6176130c58dac7d941f61: baseline-arm 2025-09-28 04:32:52,055 - lava_callback - ERROR - Error processing callback for node 68d6176130c58dac7d941f61: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:32:52,668 - lava_callback - INFO - Received callback for node 68d6176f30c58dac7d941f88 INFO: 172.21.0.1:35978 - "POST /node/68d6176f30c58dac7d941f88 HTTP/1.0" 202 Accepted 2025-09-28 04:32:52,673 - lava_callback - INFO - Starting async job submit for node 68d6176f30c58dac7d941f88 2025-09-28 04:32:52,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:32:52,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6176f30c58dac7d941f88 HTTP/1.1" 200 None 2025-09-28 04:32:52,706 - lava_callback - DEBUG - Processing node 68d6176f30c58dac7d941f88: baseline-arm 2025-09-28 04:32:52,706 - lava_callback - ERROR - Error processing callback for node 68d6176f30c58dac7d941f88: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:32:53,317 - lava_callback - INFO - Received callback for node 68d6176f30c58dac7d941f8b INFO: 172.21.0.1:35988 - "POST /node/68d6176f30c58dac7d941f8b HTTP/1.0" 202 Accepted 2025-09-28 04:32:53,323 - lava_callback - INFO - Starting async job submit for node 68d6176f30c58dac7d941f8b 2025-09-28 04:32:53,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:32:53,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6176f30c58dac7d941f8b HTTP/1.1" 200 None 2025-09-28 04:32:53,340 - lava_callback - DEBUG - Processing node 68d6176f30c58dac7d941f8b: baseline-arm 2025-09-28 04:32:53,340 - lava_callback - ERROR - Error processing callback for node 68d6176f30c58dac7d941f8b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:32:53,961 - lava_callback - INFO - Received callback for node 68d6177130c58dac7d941f8e INFO: 172.21.0.1:36004 - "POST /node/68d6177130c58dac7d941f8e HTTP/1.0" 202 Accepted 2025-09-28 04:32:53,967 - lava_callback - INFO - Starting async job submit for node 68d6177130c58dac7d941f8e 2025-09-28 04:32:53,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:32:53,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6177130c58dac7d941f8e HTTP/1.1" 200 None 2025-09-28 04:32:53,980 - lava_callback - DEBUG - Processing node 68d6177130c58dac7d941f8e: baseline-arm 2025-09-28 04:32:53,980 - lava_callback - ERROR - Error processing callback for node 68d6177130c58dac7d941f8e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:32:54,618 - lava_callback - INFO - Received callback for node 68d6177130c58dac7d941f91 INFO: 172.21.0.1:36018 - "POST /node/68d6177130c58dac7d941f91 HTTP/1.0" 202 Accepted 2025-09-28 04:32:54,624 - lava_callback - INFO - Starting async job submit for node 68d6177130c58dac7d941f91 2025-09-28 04:32:54,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:32:54,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6177130c58dac7d941f91 HTTP/1.1" 200 None 2025-09-28 04:32:54,637 - lava_callback - DEBUG - Processing node 68d6177130c58dac7d941f91: baseline-arm 2025-09-28 04:32:54,637 - lava_callback - ERROR - Error processing callback for node 68d6177130c58dac7d941f91: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39632 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:33:53,810 - lava_callback - INFO - Received callback for node 68d617a130c58dac7d941fea INFO: 172.21.0.1:45028 - "POST /node/68d617a130c58dac7d941fea HTTP/1.0" 202 Accepted 2025-09-28 04:33:53,816 - lava_callback - INFO - Starting async job submit for node 68d617a130c58dac7d941fea 2025-09-28 04:33:53,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:33:53,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d617a130c58dac7d941fea HTTP/1.1" 200 None 2025-09-28 04:33:53,837 - lava_callback - DEBUG - Processing node 68d617a130c58dac7d941fea: ltp-watchqueue 2025-09-28 04:33:53,838 - lava_callback - ERROR - Error processing callback for node 68d617a130c58dac7d941fea: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:33:54,599 - lava_callback - INFO - Received callback for node 68d617a230c58dac7d941fed INFO: 172.21.0.1:45044 - "POST /node/68d617a230c58dac7d941fed HTTP/1.0" 202 Accepted 2025-09-28 04:33:54,608 - lava_callback - INFO - Starting async job submit for node 68d617a230c58dac7d941fed 2025-09-28 04:33:54,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:33:54,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d617a230c58dac7d941fed HTTP/1.1" 200 None 2025-09-28 04:33:54,622 - lava_callback - DEBUG - Processing node 68d617a230c58dac7d941fed: ltp-timers 2025-09-28 04:33:54,622 - lava_callback - ERROR - Error processing callback for node 68d617a230c58dac7d941fed: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35498 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:34:08,102 - lava_callback - INFO - Received callback for node 68d8b643a178a5bbc0ea6af1 INFO: 172.21.0.1:35514 - "POST /node/68d8b643a178a5bbc0ea6af1 HTTP/1.0" 202 Accepted 2025-09-28 04:34:08,109 - lava_callback - INFO - Starting async job submit for node 68d8b643a178a5bbc0ea6af1 2025-09-28 04:34:08,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:08,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b643a178a5bbc0ea6af1 HTTP/1.1" 200 None 2025-09-28 04:34:08,153 - lava_callback - DEBUG - Processing node 68d8b643a178a5bbc0ea6af1: baseline-arm 2025-09-28 04:34:08,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:34:08,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:08,639 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b643a178a5bbc0ea6af1/log.txt.gz 2025-09-28 04:34:08,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:34:08,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:08,768 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b643a178a5bbc0ea6af1/lava_callback.json.gz DEBUG: Setup failed for 68d8b643a178a5bbc0ea6af1. Transitting job node result: incomplete -> fail 2025-09-28 04:34:08,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:08,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b643a178a5bbc0ea6af1 HTTP/1.1" 200 None 2025-09-28 04:34:08,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:08,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:34:08,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:08,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b643a178a5bbc0ea6af1 HTTP/1.1" 200 None 2025-09-28 04:34:08,880 - lava_callback - INFO - Completed processing callback for node 68d8b643a178a5bbc0ea6af1 INFO: 172.21.0.1:35528 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:34:15,143 - lava_callback - INFO - Received callback for node 68d8b5dca178a5bbc0ea6a2a INFO: 172.21.0.1:42196 - "POST /node/68d8b5dca178a5bbc0ea6a2a HTTP/1.0" 202 Accepted 2025-09-28 04:34:15,150 - lava_callback - INFO - Starting async job submit for node 68d8b5dca178a5bbc0ea6a2a INFO: 172.21.0.1:42198 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 04:34:15,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:15,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dca178a5bbc0ea6a2a HTTP/1.1" 200 None 2025-09-28 04:34:15,204 - lava_callback - DEBUG - Processing node 68d8b5dca178a5bbc0ea6a2a: baseline-arm64 2025-09-28 04:34:16,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:34:16,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:16,486 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dca178a5bbc0ea6a2a/log.txt.gz 2025-09-28 04:34:16,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42206 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:34:16,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:16,670 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dca178a5bbc0ea6a2a/lava_callback.json.gz DEBUG: Setup failed for 68d8b5dca178a5bbc0ea6a2a. Transitting job node result: incomplete -> fail 2025-09-28 04:34:16,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:16,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dca178a5bbc0ea6a2a HTTP/1.1" 200 None 2025-09-28 04:34:16,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:16,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:34:16,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:16,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dca178a5bbc0ea6a2a HTTP/1.1" 200 None 2025-09-28 04:34:16,803 - lava_callback - INFO - Completed processing callback for node 68d8b5dca178a5bbc0ea6a2a INFO: 172.21.0.1:46882 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:34:31,761 - lava_callback - INFO - Received callback for node 68d8b5dba178a5bbc0ea6a24 INFO: 172.21.0.1:59094 - "POST /node/68d8b5dba178a5bbc0ea6a24 HTTP/1.0" 202 Accepted 2025-09-28 04:34:31,769 - lava_callback - INFO - Starting async job submit for node 68d8b5dba178a5bbc0ea6a24 2025-09-28 04:34:31,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:31,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dba178a5bbc0ea6a24 HTTP/1.1" 200 None 2025-09-28 04:34:31,822 - lava_callback - DEBUG - Processing node 68d8b5dba178a5bbc0ea6a24: baseline-arm64 2025-09-28 04:34:32,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:34:33,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:33,100 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dba178a5bbc0ea6a24/log.txt.gz 2025-09-28 04:34:33,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:34:33,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:33,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dba178a5bbc0ea6a24/lava_callback.json.gz DEBUG: Setup failed for 68d8b5dba178a5bbc0ea6a24. Transitting job node result: incomplete -> fail 2025-09-28 04:34:33,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:33,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dba178a5bbc0ea6a24 HTTP/1.1" 200 None 2025-09-28 04:34:33,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:33,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:34:33,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:33,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dba178a5bbc0ea6a24 HTTP/1.1" 200 None 2025-09-28 04:34:33,437 - lava_callback - INFO - Completed processing callback for node 68d8b5dba178a5bbc0ea6a24 2025-09-28 04:34:33,725 - lava_callback - INFO - Received callback for node 68d8b5e0a178a5bbc0ea6a45 INFO: 172.21.0.1:59106 - "POST /node/68d8b5e0a178a5bbc0ea6a45 HTTP/1.0" 202 Accepted 2025-09-28 04:34:33,733 - lava_callback - INFO - Starting async job submit for node 68d8b5e0a178a5bbc0ea6a45 2025-09-28 04:34:33,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:33,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e0a178a5bbc0ea6a45 HTTP/1.1" 200 None 2025-09-28 04:34:33,792 - lava_callback - DEBUG - Processing node 68d8b5e0a178a5bbc0ea6a45: baseline-arm64 2025-09-28 04:34:35,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:34:35,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:35,314 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e0a178a5bbc0ea6a45/log.txt.gz 2025-09-28 04:34:35,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:34:35,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:34:35,504 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5e0a178a5bbc0ea6a45/lava_callback.json.gz DEBUG: Setup failed for 68d8b5e0a178a5bbc0ea6a45. Transitting job node result: incomplete -> fail 2025-09-28 04:34:35,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:35,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e0a178a5bbc0ea6a45 HTTP/1.1" 200 None 2025-09-28 04:34:35,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:35,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:34:35,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:34:35,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e0a178a5bbc0ea6a45 HTTP/1.1" 200 None 2025-09-28 04:34:35,667 - lava_callback - INFO - Completed processing callback for node 68d8b5e0a178a5bbc0ea6a45 INFO: 172.21.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:35:41,788 - lava_callback - INFO - Received callback for node 68d8b6aaa178a5bbc0ea6b59 INFO: 172.21.0.1:58680 - "POST /node/68d8b6aaa178a5bbc0ea6b59 HTTP/1.0" 202 Accepted 2025-09-28 04:35:41,795 - lava_callback - INFO - Starting async job submit for node 68d8b6aaa178a5bbc0ea6b59 2025-09-28 04:35:41,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:35:41,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b6aaa178a5bbc0ea6b59 HTTP/1.1" 200 None 2025-09-28 04:35:41,884 - lava_callback - DEBUG - Processing node 68d8b6aaa178a5bbc0ea6b59: ltp-smoketest 2025-09-28 04:35:42,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:35:42,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:35:42,504 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8b6aaa178a5bbc0ea6b59/log.txt.gz 2025-09-28 04:35:42,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:35:42,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:35:42,654 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8b6aaa178a5bbc0ea6b59/lava_callback.json.gz 2025-09-28 04:35:42,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:35:42,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b6aaa178a5bbc0ea6b59 HTTP/1.1" 200 None 2025-09-28 04:35:42,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:35:42,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae6a178a5bbc0ea5cb4 HTTP/1.1" 200 None 2025-09-28 04:35:42,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:35:42,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b6aaa178a5bbc0ea6b59 HTTP/1.1" 200 None 2025-09-28 04:35:42,833 - lava_callback - INFO - Completed processing callback for node 68d8b6aaa178a5bbc0ea6b59 INFO: 172.21.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58308 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:38:19,137 - lava_callback - INFO - Received callback for node 68d8b648a178a5bbc0ea6af8 INFO: 172.21.0.1:58312 - "POST /node/68d8b648a178a5bbc0ea6af8 HTTP/1.0" 202 Accepted 2025-09-28 04:38:19,144 - lava_callback - INFO - Starting async job submit for node 68d8b648a178a5bbc0ea6af8 2025-09-28 04:38:19,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:38:19,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b648a178a5bbc0ea6af8 HTTP/1.1" 200 None 2025-09-28 04:38:19,198 - lava_callback - DEBUG - Processing node 68d8b648a178a5bbc0ea6af8: baseline-x86 2025-09-28 04:38:19,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:38:20,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:38:20,001 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b648a178a5bbc0ea6af8/log.txt.gz 2025-09-28 04:38:20,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:38:20,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:38:20,205 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b648a178a5bbc0ea6af8/lava_callback.json.gz DEBUG: Setup failed for 68d8b648a178a5bbc0ea6af8. Transitting job node result: incomplete -> fail 2025-09-28 04:38:20,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:38:20,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b648a178a5bbc0ea6af8 HTTP/1.1" 200 None 2025-09-28 04:38:20,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:38:20,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aafca178a5bbc0ea5cd2 HTTP/1.1" 200 None 2025-09-28 04:38:20,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:38:20,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b648a178a5bbc0ea6af8 HTTP/1.1" 200 None 2025-09-28 04:38:20,369 - lava_callback - INFO - Completed processing callback for node 68d8b648a178a5bbc0ea6af8 INFO: 172.21.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34824 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:41:12,882 - lava_callback - INFO - Received callback for node 68d6179830c58dac7d941fdb INFO: 172.21.0.1:34838 - "POST /node/68d6179830c58dac7d941fdb HTTP/1.0" 202 Accepted 2025-09-28 04:41:12,889 - lava_callback - INFO - Starting async job submit for node 68d6179830c58dac7d941fdb 2025-09-28 04:41:12,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:12,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179830c58dac7d941fdb HTTP/1.1" 200 None 2025-09-28 04:41:12,930 - lava_callback - DEBUG - Processing node 68d6179830c58dac7d941fdb: ltp-sched 2025-09-28 04:41:13,277 - lava_callback - ERROR - Error processing callback for node 68d6179830c58dac7d941fdb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:41:23,301 - lava_callback - INFO - Received callback for node 68d7fdab8dd6b39fd1aeb855 INFO: 172.21.0.1:47252 - "POST /node/68d7fdab8dd6b39fd1aeb855 HTTP/1.0" 202 Accepted 2025-09-28 04:41:23,308 - lava_callback - INFO - Starting async job submit for node 68d7fdab8dd6b39fd1aeb855 2025-09-28 04:41:23,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:23,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdab8dd6b39fd1aeb855 HTTP/1.1" 200 None 2025-09-28 04:41:23,361 - lava_callback - DEBUG - Processing node 68d7fdab8dd6b39fd1aeb855: baseline-arm64 2025-09-28 04:41:24,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:41:24,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:41:24,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdab8dd6b39fd1aeb855/log.txt.gz 2025-09-28 04:41:24,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:41:24,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:41:24,690 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdab8dd6b39fd1aeb855/lava_callback.json.gz DEBUG: Setup failed for 68d7fdab8dd6b39fd1aeb855. Transitting job node result: incomplete -> fail 2025-09-28 04:41:24,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:24,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdab8dd6b39fd1aeb855 HTTP/1.1" 200 None 2025-09-28 04:41:24,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:24,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-28 04:41:24,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:24,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fdab8dd6b39fd1aeb855 HTTP/1.1" 200 None 2025-09-28 04:41:24,834 - lava_callback - INFO - Completed processing callback for node 68d7fdab8dd6b39fd1aeb855 INFO: 172.21.0.1:47266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44234 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:41:48,472 - lava_callback - INFO - Received callback for node 68d8b173a178a5bbc0ea6318 INFO: 172.21.0.1:44246 - "POST /node/68d8b173a178a5bbc0ea6318 HTTP/1.0" 202 Accepted 2025-09-28 04:41:48,520 - lava_callback - INFO - Starting async job submit for node 68d8b173a178a5bbc0ea6318 2025-09-28 04:41:48,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:48,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b173a178a5bbc0ea6318 HTTP/1.1" 200 None 2025-09-28 04:41:48,575 - lava_callback - DEBUG - Processing node 68d8b173a178a5bbc0ea6318: job-gcc-12-x86-612-cip 2025-09-28 04:41:48,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:41:48,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:41:48,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/job-gcc-12-x86-612-cip-68d8b173a178a5bbc0ea6318/log.txt.gz 2025-09-28 04:41:48,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:41:48,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:41:48,961 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/job-gcc-12-x86-612-cip-68d8b173a178a5bbc0ea6318/lava_callback.json.gz 2025-09-28 04:41:48,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:48,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b173a178a5bbc0ea6318 HTTP/1.1" 200 None 2025-09-28 04:41:49,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:49,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f277aff7facd9a8d0f0 HTTP/1.1" 200 None 2025-09-28 04:41:49,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:41:49,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b173a178a5bbc0ea6318 HTTP/1.1" 200 None 2025-09-28 04:41:49,085 - lava_callback - INFO - Completed processing callback for node 68d8b173a178a5bbc0ea6318 INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37370 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:42:59,012 - lava_callback - INFO - Received callback for node 68d8b970a178a5bbc0ea6c4b INFO: 172.21.0.1:37378 - "POST /node/68d8b970a178a5bbc0ea6c4b HTTP/1.0" 202 Accepted 2025-09-28 04:42:59,020 - lava_callback - INFO - Starting async job submit for node 68d8b970a178a5bbc0ea6c4b 2025-09-28 04:42:59,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:42:59,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b970a178a5bbc0ea6c4b HTTP/1.1" 200 None 2025-09-28 04:42:59,071 - lava_callback - DEBUG - Processing node 68d8b970a178a5bbc0ea6c4b: baseline-arm64 2025-09-28 04:43:00,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:43:00,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:43:00,289 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b970a178a5bbc0ea6c4b/log.txt.gz 2025-09-28 04:43:00,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:43:00,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:43:00,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b970a178a5bbc0ea6c4b/lava_callback.json.gz 2025-09-28 04:43:00,553 - lava_callback - INFO - Received callback for node 68d8b96fa178a5bbc0ea6c44 DEBUG: Setup failed for 68d8b970a178a5bbc0ea6c4b. Transitting job node result: incomplete -> fail INFO: 172.21.0.1:34216 - "POST /node/68d8b96fa178a5bbc0ea6c44 HTTP/1.0" 202 Accepted 2025-09-28 04:43:00,582 - lava_callback - INFO - Starting async job submit for node 68d8b96fa178a5bbc0ea6c44 2025-09-28 04:43:00,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:00,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:00,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b970a178a5bbc0ea6c4b HTTP/1.1" 200 None 2025-09-28 04:43:00,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:00,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b96fa178a5bbc0ea6c44 HTTP/1.1" 200 None 2025-09-28 04:43:00,635 - lava_callback - DEBUG - Processing node 68d8b96fa178a5bbc0ea6c44: baseline-arm64 2025-09-28 04:43:00,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:43:00,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:00,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b970a178a5bbc0ea6c4b HTTP/1.1" 200 None 2025-09-28 04:43:00,835 - lava_callback - INFO - Completed processing callback for node 68d8b970a178a5bbc0ea6c4b 2025-09-28 04:43:01,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:43:02,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:43:02,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b96fa178a5bbc0ea6c44/log.txt.gz 2025-09-28 04:43:02,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:43:02,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:43:02,208 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b96fa178a5bbc0ea6c44/lava_callback.json.gz DEBUG: Setup failed for 68d8b96fa178a5bbc0ea6c44. Transitting job node result: incomplete -> fail 2025-09-28 04:43:02,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:02,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b96fa178a5bbc0ea6c44 HTTP/1.1" 200 None 2025-09-28 04:43:02,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:02,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:43:02,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:02,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b96fa178a5bbc0ea6c44 HTTP/1.1" 200 None 2025-09-28 04:43:02,345 - lava_callback - INFO - Completed processing callback for node 68d8b96fa178a5bbc0ea6c44 INFO: 172.21.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57204 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:43:32,849 - lava_callback - INFO - Received callback for node 68d61a5930c58dac7d9421d7 INFO: 172.21.0.1:45350 - "POST /node/68d61a5930c58dac7d9421d7 HTTP/1.0" 202 Accepted 2025-09-28 04:43:32,855 - lava_callback - INFO - Starting async job submit for node 68d61a5930c58dac7d9421d7 2025-09-28 04:43:32,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:32,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a5930c58dac7d9421d7 HTTP/1.1" 200 None 2025-09-28 04:43:32,889 - lava_callback - DEBUG - Processing node 68d61a5930c58dac7d9421d7: baseline-arm-broonie 2025-09-28 04:43:33,076 - lava_callback - ERROR - Error processing callback for node 68d61a5930c58dac7d9421d7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45360 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:43:41,989 - lava_callback - INFO - Received callback for node 68d8af28a178a5bbc0ea62b0 INFO: 172.21.0.1:44162 - "POST /node/68d8af28a178a5bbc0ea62b0 HTTP/1.0" 202 Accepted 2025-09-28 04:43:41,995 - lava_callback - INFO - Starting async job submit for node 68d8af28a178a5bbc0ea62b0 2025-09-28 04:43:42,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:42,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8af28a178a5bbc0ea62b0 HTTP/1.1" 200 None 2025-09-28 04:43:42,054 - lava_callback - DEBUG - Processing node 68d8af28a178a5bbc0ea62b0: baseline-arm 2025-09-28 04:43:42,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:43:42,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:43:42,444 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8af28a178a5bbc0ea62b0/log.txt.gz 2025-09-28 04:43:42,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:43:42,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:43:42,565 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8af28a178a5bbc0ea62b0/lava_callback.json.gz DEBUG: Setup failed for 68d8af28a178a5bbc0ea62b0. Transitting job node result: incomplete -> fail 2025-09-28 04:43:42,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:42,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8af28a178a5bbc0ea62b0 HTTP/1.1" 200 None 2025-09-28 04:43:42,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:42,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:43:42,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:43:42,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8af28a178a5bbc0ea62b0 HTTP/1.1" 200 None 2025-09-28 04:43:42,678 - lava_callback - INFO - Completed processing callback for node 68d8af28a178a5bbc0ea62b0 INFO: 172.21.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44168 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:44:36,772 - lava_callback - INFO - Received callback for node 68d8a8847aff7facd9a8d613 INFO: 172.21.0.1:44176 - "POST /node/68d8a8847aff7facd9a8d613 HTTP/1.0" 202 Accepted 2025-09-28 04:44:36,778 - lava_callback - INFO - Starting async job submit for node 68d8a8847aff7facd9a8d613 2025-09-28 04:44:36,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:36,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8847aff7facd9a8d613 HTTP/1.1" 200 None 2025-09-28 04:44:36,812 - lava_callback - DEBUG - Processing node 68d8a8847aff7facd9a8d613: baseline-arm64 2025-09-28 04:44:37,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:44:37,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:44:37,204 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8847aff7facd9a8d613/log.txt.gz 2025-09-28 04:44:37,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:44:37,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:44:37,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8847aff7facd9a8d613/lava_callback.json.gz 2025-09-28 04:44:37,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:37,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8847aff7facd9a8d613 HTTP/1.1" 200 None 2025-09-28 04:44:37,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:37,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 04:44:37,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:37,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a8847aff7facd9a8d613 HTTP/1.1" 200 None 2025-09-28 04:44:37,449 - lava_callback - INFO - Completed processing callback for node 68d8a8847aff7facd9a8d613 INFO: 172.21.0.1:51864 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:44:51,296 - lava_callback - INFO - Received callback for node 68d61a3530c58dac7d942177 INFO: 172.21.0.1:56528 - "POST /node/68d61a3530c58dac7d942177 HTTP/1.0" 202 Accepted 2025-09-28 04:44:51,301 - lava_callback - INFO - Starting async job submit for node 68d61a3530c58dac7d942177 2025-09-28 04:44:51,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:51,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a3530c58dac7d942177 HTTP/1.1" 200 None 2025-09-28 04:44:51,314 - lava_callback - DEBUG - Processing node 68d61a3530c58dac7d942177: baseline-arm 2025-09-28 04:44:51,314 - lava_callback - ERROR - Error processing callback for node 68d61a3530c58dac7d942177: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:44:51,950 - lava_callback - INFO - Received callback for node 68d61a3530c58dac7d94217a INFO: 172.21.0.1:56542 - "POST /node/68d61a3530c58dac7d94217a HTTP/1.0" 202 Accepted 2025-09-28 04:44:51,956 - lava_callback - INFO - Starting async job submit for node 68d61a3530c58dac7d94217a 2025-09-28 04:44:51,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:51,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a3530c58dac7d94217a HTTP/1.1" 200 None 2025-09-28 04:44:51,970 - lava_callback - DEBUG - Processing node 68d61a3530c58dac7d94217a: baseline-arm 2025-09-28 04:44:51,970 - lava_callback - ERROR - Error processing callback for node 68d61a3530c58dac7d94217a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:44:52,596 - lava_callback - INFO - Received callback for node 68d61a3730c58dac7d94217d INFO: 172.21.0.1:56544 - "POST /node/68d61a3730c58dac7d94217d HTTP/1.0" 202 Accepted 2025-09-28 04:44:52,603 - lava_callback - INFO - Starting async job submit for node 68d61a3730c58dac7d94217d 2025-09-28 04:44:52,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:52,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a3730c58dac7d94217d HTTP/1.1" 200 None 2025-09-28 04:44:52,617 - lava_callback - DEBUG - Processing node 68d61a3730c58dac7d94217d: baseline-arm 2025-09-28 04:44:52,617 - lava_callback - ERROR - Error processing callback for node 68d61a3730c58dac7d94217d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:44:53,246 - lava_callback - INFO - Received callback for node 68d61a3830c58dac7d942180 INFO: 172.21.0.1:56560 - "POST /node/68d61a3830c58dac7d942180 HTTP/1.0" 202 Accepted 2025-09-28 04:44:53,251 - lava_callback - INFO - Starting async job submit for node 68d61a3830c58dac7d942180 2025-09-28 04:44:53,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:44:53,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a3830c58dac7d942180 HTTP/1.1" 200 None 2025-09-28 04:44:53,287 - lava_callback - DEBUG - Processing node 68d61a3830c58dac7d942180: baseline-arm 2025-09-28 04:44:53,287 - lava_callback - ERROR - Error processing callback for node 68d61a3830c58dac7d942180: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34856 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:45:11,813 - lava_callback - INFO - Received callback for node 68d61a4530c58dac7d9421a7 INFO: 172.21.0.1:34858 - "POST /node/68d61a4530c58dac7d9421a7 HTTP/1.0" 202 Accepted 2025-09-28 04:45:11,818 - lava_callback - INFO - Starting async job submit for node 68d61a4530c58dac7d9421a7 2025-09-28 04:45:11,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:11,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a4530c58dac7d9421a7 HTTP/1.1" 200 None 2025-09-28 04:45:11,832 - lava_callback - DEBUG - Processing node 68d61a4530c58dac7d9421a7: baseline-arm 2025-09-28 04:45:11,832 - lava_callback - ERROR - Error processing callback for node 68d61a4530c58dac7d9421a7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:12,483 - lava_callback - INFO - Received callback for node 68d61a4630c58dac7d9421aa INFO: 172.21.0.1:34866 - "POST /node/68d61a4630c58dac7d9421aa HTTP/1.0" 202 Accepted 2025-09-28 04:45:12,489 - lava_callback - INFO - Starting async job submit for node 68d61a4630c58dac7d9421aa 2025-09-28 04:45:12,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:12,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a4630c58dac7d9421aa HTTP/1.1" 200 None 2025-09-28 04:45:12,502 - lava_callback - DEBUG - Processing node 68d61a4630c58dac7d9421aa: baseline-arm 2025-09-28 04:45:12,502 - lava_callback - ERROR - Error processing callback for node 68d61a4630c58dac7d9421aa: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:13,138 - lava_callback - INFO - Received callback for node 68d61a4730c58dac7d9421ad INFO: 172.21.0.1:34868 - "POST /node/68d61a4730c58dac7d9421ad HTTP/1.0" 202 Accepted 2025-09-28 04:45:13,143 - lava_callback - INFO - Starting async job submit for node 68d61a4730c58dac7d9421ad 2025-09-28 04:45:13,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:13,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a4730c58dac7d9421ad HTTP/1.1" 200 None 2025-09-28 04:45:13,164 - lava_callback - DEBUG - Processing node 68d61a4730c58dac7d9421ad: baseline-arm 2025-09-28 04:45:13,164 - lava_callback - ERROR - Error processing callback for node 68d61a4730c58dac7d9421ad: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:13,795 - lava_callback - INFO - Received callback for node 68d61a4830c58dac7d9421b0 INFO: 172.21.0.1:34878 - "POST /node/68d61a4830c58dac7d9421b0 HTTP/1.0" 202 Accepted 2025-09-28 04:45:13,801 - lava_callback - INFO - Starting async job submit for node 68d61a4830c58dac7d9421b0 2025-09-28 04:45:13,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:13,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a4830c58dac7d9421b0 HTTP/1.1" 200 None 2025-09-28 04:45:13,816 - lava_callback - DEBUG - Processing node 68d61a4830c58dac7d9421b0: baseline-arm 2025-09-28 04:45:13,816 - lava_callback - ERROR - Error processing callback for node 68d61a4830c58dac7d9421b0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42822 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:45:34,581 - lava_callback - INFO - Received callback for node 68d61a5e30c58dac7d9421e0 INFO: 172.21.0.1:53124 - "POST /node/68d61a5e30c58dac7d9421e0 HTTP/1.0" 202 Accepted 2025-09-28 04:45:34,587 - lava_callback - INFO - Starting async job submit for node 68d61a5e30c58dac7d9421e0 2025-09-28 04:45:34,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:34,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a5e30c58dac7d9421e0 HTTP/1.1" 200 None 2025-09-28 04:45:34,600 - lava_callback - DEBUG - Processing node 68d61a5e30c58dac7d9421e0: baseline-arm-broonie 2025-09-28 04:45:34,600 - lava_callback - ERROR - Error processing callback for node 68d61a5e30c58dac7d9421e0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53130 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:45:44,413 - lava_callback - INFO - Received callback for node 68d61a6330c58dac7d9421e9 INFO: 172.21.0.1:59312 - "POST /node/68d61a6330c58dac7d9421e9 HTTP/1.0" 202 Accepted 2025-09-28 04:45:44,418 - lava_callback - INFO - Starting async job submit for node 68d61a6330c58dac7d9421e9 2025-09-28 04:45:44,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:44,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a6330c58dac7d9421e9 HTTP/1.1" 200 None 2025-09-28 04:45:44,430 - lava_callback - DEBUG - Processing node 68d61a6330c58dac7d9421e9: baseline-arm-clabbe 2025-09-28 04:45:44,430 - lava_callback - ERROR - Error processing callback for node 68d61a6330c58dac7d9421e9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:45,077 - lava_callback - INFO - Received callback for node 68d61a6530c58dac7d9421ec INFO: 172.21.0.1:59326 - "POST /node/68d61a6530c58dac7d9421ec HTTP/1.0" 202 Accepted 2025-09-28 04:45:45,082 - lava_callback - INFO - Starting async job submit for node 68d61a6530c58dac7d9421ec 2025-09-28 04:45:45,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:45,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a6530c58dac7d9421ec HTTP/1.1" 200 None 2025-09-28 04:45:45,230 - lava_callback - DEBUG - Processing node 68d61a6530c58dac7d9421ec: baseline-arm-clabbe 2025-09-28 04:45:45,231 - lava_callback - ERROR - Error processing callback for node 68d61a6530c58dac7d9421ec: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:45,732 - lava_callback - INFO - Received callback for node 68d61a6930c58dac7d9421f2 INFO: 172.21.0.1:59336 - "POST /node/68d61a6930c58dac7d9421f2 HTTP/1.0" 202 Accepted 2025-09-28 04:45:45,737 - lava_callback - INFO - Starting async job submit for node 68d61a6930c58dac7d9421f2 2025-09-28 04:45:45,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:45,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a6930c58dac7d9421f2 HTTP/1.1" 200 None 2025-09-28 04:45:45,751 - lava_callback - DEBUG - Processing node 68d61a6930c58dac7d9421f2: baseline-arm-clabbe 2025-09-28 04:45:45,751 - lava_callback - ERROR - Error processing callback for node 68d61a6930c58dac7d9421f2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:46,378 - lava_callback - INFO - Received callback for node 68d61a6a30c58dac7d9421f5 INFO: 172.21.0.1:59346 - "POST /node/68d61a6a30c58dac7d9421f5 HTTP/1.0" 202 Accepted 2025-09-28 04:45:46,384 - lava_callback - INFO - Starting async job submit for node 68d61a6a30c58dac7d9421f5 2025-09-28 04:45:46,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:46,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a6a30c58dac7d9421f5 HTTP/1.1" 200 None 2025-09-28 04:45:46,397 - lava_callback - DEBUG - Processing node 68d61a6a30c58dac7d9421f5: baseline-arm-clabbe 2025-09-28 04:45:46,397 - lava_callback - ERROR - Error processing callback for node 68d61a6a30c58dac7d9421f5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:45:47,043 - lava_callback - INFO - Received callback for node 68d61a6c30c58dac7d9421f8 INFO: 172.21.0.1:59370 - "POST /node/68d61a6c30c58dac7d9421f8 HTTP/1.0" 202 Accepted 2025-09-28 04:45:47,049 - lava_callback - INFO - Starting async job submit for node 68d61a6c30c58dac7d9421f8 2025-09-28 04:45:47,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:47,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a6c30c58dac7d9421f8 HTTP/1.1" 200 None 2025-09-28 04:45:47,062 - lava_callback - DEBUG - Processing node 68d61a6c30c58dac7d9421f8: baseline-arm-clabbe 2025-09-28 04:45:47,062 - lava_callback - ERROR - Error processing callback for node 68d61a6c30c58dac7d9421f8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:47,706 - lava_callback - INFO - Received callback for node 68d61a7330c58dac7d942212 INFO: 172.21.0.1:59380 - "POST /node/68d61a7330c58dac7d942212 HTTP/1.0" 202 Accepted 2025-09-28 04:45:47,712 - lava_callback - INFO - Starting async job submit for node 68d61a7330c58dac7d942212 2025-09-28 04:45:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:47,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a7330c58dac7d942212 HTTP/1.1" 200 None 2025-09-28 04:45:47,725 - lava_callback - DEBUG - Processing node 68d61a7330c58dac7d942212: baseline-arm-clabbe 2025-09-28 04:45:47,725 - lava_callback - ERROR - Error processing callback for node 68d61a7330c58dac7d942212: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:54,440 - lava_callback - INFO - Received callback for node 68d61a7d30c58dac7d942221 INFO: 172.21.0.1:48492 - "POST /node/68d61a7d30c58dac7d942221 HTTP/1.0" 202 Accepted 2025-09-28 04:45:54,446 - lava_callback - INFO - Starting async job submit for node 68d61a7d30c58dac7d942221 2025-09-28 04:45:54,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:54,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a7d30c58dac7d942221 HTTP/1.1" 200 None 2025-09-28 04:45:54,459 - lava_callback - DEBUG - Processing node 68d61a7d30c58dac7d942221: ltp-containers 2025-09-28 04:45:54,459 - lava_callback - ERROR - Error processing callback for node 68d61a7d30c58dac7d942221: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:55,170 - lava_callback - INFO - Received callback for node 68d61a7e30c58dac7d942224 INFO: 172.21.0.1:48504 - "POST /node/68d61a7e30c58dac7d942224 HTTP/1.0" 202 Accepted 2025-09-28 04:45:55,177 - lava_callback - INFO - Starting async job submit for node 68d61a7e30c58dac7d942224 2025-09-28 04:45:55,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:55,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a7e30c58dac7d942224 HTTP/1.1" 200 None 2025-09-28 04:45:55,193 - lava_callback - DEBUG - Processing node 68d61a7e30c58dac7d942224: ltp-controllers 2025-09-28 04:45:55,193 - lava_callback - ERROR - Error processing callback for node 68d61a7e30c58dac7d942224: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:45:55,945 - lava_callback - INFO - Received callback for node 68d61a8030c58dac7d942227 INFO: 172.21.0.1:48512 - "POST /node/68d61a8030c58dac7d942227 HTTP/1.0" 202 Accepted 2025-09-28 04:45:55,951 - lava_callback - INFO - Starting async job submit for node 68d61a8030c58dac7d942227 2025-09-28 04:45:55,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:45:55,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8030c58dac7d942227 HTTP/1.1" 200 None 2025-09-28 04:45:55,967 - lava_callback - DEBUG - Processing node 68d61a8030c58dac7d942227: ltp-cve 2025-09-28 04:45:55,967 - lava_callback - ERROR - Error processing callback for node 68d61a8030c58dac7d942227: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57642 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:46:14,454 - lava_callback - INFO - Received callback for node 68d61a8430c58dac7d94222d INFO: 172.21.0.1:57648 - "POST /node/68d61a8430c58dac7d94222d HTTP/1.0" 202 Accepted 2025-09-28 04:46:14,460 - lava_callback - INFO - Starting async job submit for node 68d61a8430c58dac7d94222d 2025-09-28 04:46:14,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:14,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8430c58dac7d94222d HTTP/1.1" 200 None 2025-09-28 04:46:14,473 - lava_callback - DEBUG - Processing node 68d61a8430c58dac7d94222d: ltp-fs-bind 2025-09-28 04:46:14,473 - lava_callback - ERROR - Error processing callback for node 68d61a8430c58dac7d94222d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57660 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 04:46:15,201 - lava_callback - INFO - Received callback for node 68d61a8530c58dac7d942230 INFO: 172.21.0.1:57662 - "POST /node/68d61a8530c58dac7d942230 HTTP/1.0" 202 Accepted 2025-09-28 04:46:15,207 - lava_callback - INFO - Starting async job submit for node 68d61a8530c58dac7d942230 2025-09-28 04:46:15,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:15,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8530c58dac7d942230 HTTP/1.1" 200 None 2025-09-28 04:46:15,220 - lava_callback - DEBUG - Processing node 68d61a8530c58dac7d942230: ltp-fsx 2025-09-28 04:46:15,220 - lava_callback - ERROR - Error processing callback for node 68d61a8530c58dac7d942230: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:15,914 - lava_callback - INFO - Received callback for node 68d61a8730c58dac7d942233 INFO: 172.21.0.1:57676 - "POST /node/68d61a8730c58dac7d942233 HTTP/1.0" 202 Accepted 2025-09-28 04:46:15,920 - lava_callback - INFO - Starting async job submit for node 68d61a8730c58dac7d942233 2025-09-28 04:46:15,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:15,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8730c58dac7d942233 HTTP/1.1" 200 None 2025-09-28 04:46:15,936 - lava_callback - DEBUG - Processing node 68d61a8730c58dac7d942233: ltp-input 2025-09-28 04:46:15,936 - lava_callback - ERROR - Error processing callback for node 68d61a8730c58dac7d942233: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57688 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:46:16,537 - lava_callback - INFO - Received callback for node 68d8a8887aff7facd9a8d62e INFO: 172.21.0.1:57690 - "POST /node/68d8a8887aff7facd9a8d62e HTTP/1.0" 202 Accepted 2025-09-28 04:46:16,542 - lava_callback - INFO - Starting async job submit for node 68d8a8887aff7facd9a8d62e 2025-09-28 04:46:16,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:16,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8887aff7facd9a8d62e HTTP/1.1" 200 None 2025-09-28 04:46:16,576 - lava_callback - DEBUG - Processing node 68d8a8887aff7facd9a8d62e: baseline-arm64 2025-09-28 04:46:16,716 - lava_callback - INFO - Received callback for node 68d61a8930c58dac7d942236 INFO: 172.21.0.1:57704 - "POST /node/68d61a8930c58dac7d942236 HTTP/1.0" 202 Accepted 2025-09-28 04:46:16,732 - lava_callback - INFO - Starting async job submit for node 68d61a8930c58dac7d942236 2025-09-28 04:46:16,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:16,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8930c58dac7d942236 HTTP/1.1" 200 None 2025-09-28 04:46:16,821 - lava_callback - DEBUG - Processing node 68d61a8930c58dac7d942236: ltp-ipc 2025-09-28 04:46:16,822 - lava_callback - ERROR - Error processing callback for node 68d61a8930c58dac7d942236: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:16,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:46:16,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:46:16,977 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8887aff7facd9a8d62e/log.txt.gz 2025-09-28 04:46:16,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:46:17,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:46:17,171 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8a8887aff7facd9a8d62e/lava_callback.json.gz 2025-09-28 04:46:17,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:17,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a8887aff7facd9a8d62e HTTP/1.1" 200 None 2025-09-28 04:46:17,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:17,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f3a7aff7facd9a8d10e HTTP/1.1" 200 None 2025-09-28 04:46:17,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:17,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a8887aff7facd9a8d62e HTTP/1.1" 200 None 2025-09-28 04:46:17,246 - lava_callback - INFO - Completed processing callback for node 68d8a8887aff7facd9a8d62e 2025-09-28 04:46:17,455 - lava_callback - INFO - Received callback for node 68d61a8a30c58dac7d942239 INFO: 172.21.0.1:57712 - "POST /node/68d61a8a30c58dac7d942239 HTTP/1.0" 202 Accepted 2025-09-28 04:46:17,461 - lava_callback - INFO - Starting async job submit for node 68d61a8a30c58dac7d942239 2025-09-28 04:46:17,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:17,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8a30c58dac7d942239 HTTP/1.1" 200 None 2025-09-28 04:46:17,475 - lava_callback - DEBUG - Processing node 68d61a8a30c58dac7d942239: ltp-pty 2025-09-28 04:46:17,475 - lava_callback - ERROR - Error processing callback for node 68d61a8a30c58dac7d942239: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:18,190 - lava_callback - INFO - Received callback for node 68d61a8c30c58dac7d94223c INFO: 172.21.0.1:57714 - "POST /node/68d61a8c30c58dac7d94223c HTTP/1.0" 202 Accepted 2025-09-28 04:46:18,196 - lava_callback - INFO - Starting async job submit for node 68d61a8c30c58dac7d94223c 2025-09-28 04:46:18,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:18,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8c30c58dac7d94223c HTTP/1.1" 200 None 2025-09-28 04:46:18,210 - lava_callback - DEBUG - Processing node 68d61a8c30c58dac7d94223c: ltp-sched 2025-09-28 04:46:18,210 - lava_callback - ERROR - Error processing callback for node 68d61a8c30c58dac7d94223c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:18,892 - lava_callback - INFO - Received callback for node 68d61a8e30c58dac7d94223f INFO: 172.21.0.1:57722 - "POST /node/68d61a8e30c58dac7d94223f HTTP/1.0" 202 Accepted 2025-09-28 04:46:18,898 - lava_callback - INFO - Starting async job submit for node 68d61a8e30c58dac7d94223f 2025-09-28 04:46:18,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:18,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8e30c58dac7d94223f HTTP/1.1" 200 None 2025-09-28 04:46:18,913 - lava_callback - DEBUG - Processing node 68d61a8e30c58dac7d94223f: ltp-smoketest 2025-09-28 04:46:18,914 - lava_callback - ERROR - Error processing callback for node 68d61a8e30c58dac7d94223f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:19,625 - lava_callback - INFO - Received callback for node 68d61a8f30c58dac7d942242 INFO: 172.21.0.1:56372 - "POST /node/68d61a8f30c58dac7d942242 HTTP/1.0" 202 Accepted 2025-09-28 04:46:19,631 - lava_callback - INFO - Starting async job submit for node 68d61a8f30c58dac7d942242 2025-09-28 04:46:19,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:19,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a8f30c58dac7d942242 HTTP/1.1" 200 None 2025-09-28 04:46:19,644 - lava_callback - DEBUG - Processing node 68d61a8f30c58dac7d942242: ltp-syscalls 2025-09-28 04:46:19,644 - lava_callback - ERROR - Error processing callback for node 68d61a8f30c58dac7d942242: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:20,347 - lava_callback - INFO - Received callback for node 68d61a9130c58dac7d942245 INFO: 172.21.0.1:56384 - "POST /node/68d61a9130c58dac7d942245 HTTP/1.0" 202 Accepted 2025-09-28 04:46:20,353 - lava_callback - INFO - Starting async job submit for node 68d61a9130c58dac7d942245 2025-09-28 04:46:20,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:20,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a9130c58dac7d942245 HTTP/1.1" 200 None 2025-09-28 04:46:20,367 - lava_callback - DEBUG - Processing node 68d61a9130c58dac7d942245: ltp-syscalls-ipc 2025-09-28 04:46:20,367 - lava_callback - ERROR - Error processing callback for node 68d61a9130c58dac7d942245: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:21,061 - lava_callback - INFO - Received callback for node 68d61a9330c58dac7d942248 INFO: 172.21.0.1:56398 - "POST /node/68d61a9330c58dac7d942248 HTTP/1.0" 202 Accepted 2025-09-28 04:46:21,067 - lava_callback - INFO - Starting async job submit for node 68d61a9330c58dac7d942248 2025-09-28 04:46:21,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:21,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a9330c58dac7d942248 HTTP/1.1" 200 None 2025-09-28 04:46:21,080 - lava_callback - DEBUG - Processing node 68d61a9330c58dac7d942248: ltp-timers 2025-09-28 04:46:21,080 - lava_callback - ERROR - Error processing callback for node 68d61a9330c58dac7d942248: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:46:21,785 - lava_callback - INFO - Received callback for node 68d61a9430c58dac7d94224b INFO: 172.21.0.1:56406 - "POST /node/68d61a9430c58dac7d94224b HTTP/1.0" 202 Accepted 2025-09-28 04:46:21,792 - lava_callback - INFO - Starting async job submit for node 68d61a9430c58dac7d94224b 2025-09-28 04:46:21,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:21,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a9430c58dac7d94224b HTTP/1.1" 200 None 2025-09-28 04:46:21,806 - lava_callback - DEBUG - Processing node 68d61a9430c58dac7d94224b: ltp-watchqueue 2025-09-28 04:46:21,806 - lava_callback - ERROR - Error processing callback for node 68d61a9430c58dac7d94224b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56408 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:46:31,303 - lava_callback - INFO - Received callback for node 68d7fdad8dd6b39fd1aeb858 INFO: 172.21.0.1:51864 - "POST /node/68d7fdad8dd6b39fd1aeb858 HTTP/1.0" 202 Accepted 2025-09-28 04:46:31,310 - lava_callback - INFO - Starting async job submit for node 68d7fdad8dd6b39fd1aeb858 2025-09-28 04:46:31,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:31,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdad8dd6b39fd1aeb858 HTTP/1.1" 200 None 2025-09-28 04:46:31,396 - lava_callback - DEBUG - Processing node 68d7fdad8dd6b39fd1aeb858: baseline-arm64 2025-09-28 04:46:32,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:46:32,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:46:32,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdad8dd6b39fd1aeb858/log.txt.gz 2025-09-28 04:46:32,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:46:32,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:46:32,641 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdad8dd6b39fd1aeb858/lava_callback.json.gz DEBUG: Setup failed for 68d7fdad8dd6b39fd1aeb858. Transitting job node result: incomplete -> fail 2025-09-28 04:46:32,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:32,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdad8dd6b39fd1aeb858 HTTP/1.1" 200 None 2025-09-28 04:46:32,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:32,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-28 04:46:32,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:32,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fdad8dd6b39fd1aeb858 HTTP/1.1" 200 None 2025-09-28 04:46:32,778 - lava_callback - INFO - Completed processing callback for node 68d7fdad8dd6b39fd1aeb858 INFO: 172.21.0.1:51870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53122 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:46:51,962 - lava_callback - INFO - Received callback for node 68d7fdae8dd6b39fd1aeb85b INFO: 172.21.0.1:51906 - "POST /node/68d7fdae8dd6b39fd1aeb85b HTTP/1.0" 202 Accepted 2025-09-28 04:46:51,969 - lava_callback - INFO - Starting async job submit for node 68d7fdae8dd6b39fd1aeb85b 2025-09-28 04:46:52,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:52,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdae8dd6b39fd1aeb85b HTTP/1.1" 200 None 2025-09-28 04:46:52,027 - lava_callback - DEBUG - Processing node 68d7fdae8dd6b39fd1aeb85b: baseline-arm64 2025-09-28 04:46:52,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:46:53,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:46:53,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdae8dd6b39fd1aeb85b/log.txt.gz 2025-09-28 04:46:53,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:46:53,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:46:53,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fdae8dd6b39fd1aeb85b/lava_callback.json.gz DEBUG: Setup failed for 68d7fdae8dd6b39fd1aeb85b. Transitting job node result: incomplete -> fail 2025-09-28 04:46:53,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:53,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fdae8dd6b39fd1aeb85b HTTP/1.1" 200 None 2025-09-28 04:46:53,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:53,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d62c44c1bc82a30b37a916 HTTP/1.1" 200 None 2025-09-28 04:46:53,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:46:53,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fdae8dd6b39fd1aeb85b HTTP/1.1" 200 None 2025-09-28 04:46:53,422 - lava_callback - INFO - Completed processing callback for node 68d7fdae8dd6b39fd1aeb85b INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:47:02,335 - lava_callback - INFO - Received callback for node 68d8b95ba178a5bbc0ea6c3e INFO: 172.21.0.1:60818 - "POST /node/68d8b95ba178a5bbc0ea6c3e HTTP/1.0" 202 Accepted 2025-09-28 04:47:02,342 - lava_callback - INFO - Starting async job submit for node 68d8b95ba178a5bbc0ea6c3e 2025-09-28 04:47:02,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:47:02,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b95ba178a5bbc0ea6c3e HTTP/1.1" 200 None 2025-09-28 04:47:02,382 - lava_callback - DEBUG - Processing node 68d8b95ba178a5bbc0ea6c3e: ltp-syscalls-ipc 2025-09-28 04:47:03,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:47:03,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:47:03,347 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8b95ba178a5bbc0ea6c3e/log.txt.gz 2025-09-28 04:47:03,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:47:03,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:47:03,482 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8b95ba178a5bbc0ea6c3e/lava_callback.json.gz 2025-09-28 04:47:03,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:47:03,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b95ba178a5bbc0ea6c3e HTTP/1.1" 200 None 2025-09-28 04:47:03,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:47:03,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 04:47:03,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:47:03,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b95ba178a5bbc0ea6c3e HTTP/1.1" 200 None 2025-09-28 04:47:03,598 - lava_callback - INFO - Completed processing callback for node 68d8b95ba178a5bbc0ea6c3e INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33928 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:47:53,163 - lava_callback - INFO - Received callback for node 68d61a7b30c58dac7d94221e INFO: 172.21.0.1:51660 - "POST /node/68d61a7b30c58dac7d94221e HTTP/1.0" 202 Accepted 2025-09-28 04:47:53,169 - lava_callback - INFO - Starting async job submit for node 68d61a7b30c58dac7d94221e 2025-09-28 04:47:53,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:47:53,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61a7b30c58dac7d94221e HTTP/1.1" 200 None 2025-09-28 04:47:53,216 - lava_callback - DEBUG - Processing node 68d61a7b30c58dac7d94221e: ltp-cap-bounds 2025-09-28 04:47:53,542 - lava_callback - ERROR - Error processing callback for node 68d61a7b30c58dac7d94221e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51676 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:48:02,611 - lava_callback - INFO - Received callback for node 68d8b5e8a178a5bbc0ea6a81 INFO: 172.21.0.1:34530 - "POST /node/68d8b5e8a178a5bbc0ea6a81 HTTP/1.0" 202 Accepted 2025-09-28 04:48:02,618 - lava_callback - INFO - Starting async job submit for node 68d8b5e8a178a5bbc0ea6a81 2025-09-28 04:48:02,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:02,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e8a178a5bbc0ea6a81 HTTP/1.1" 200 None 2025-09-28 04:48:02,649 - lava_callback - DEBUG - Processing node 68d8b5e8a178a5bbc0ea6a81: baseline-arm64-broonie 2025-09-28 04:48:02,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:48:03,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:48:03,017 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e8a178a5bbc0ea6a81/log.txt.gz 2025-09-28 04:48:03,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:48:03,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:48:03,274 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8b5e8a178a5bbc0ea6a81/lava_callback.json.gz 2025-09-28 04:48:03,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:03,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5e8a178a5bbc0ea6a81 HTTP/1.1" 200 None 2025-09-28 04:48:03,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:03,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:48:03,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:03,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5e8a178a5bbc0ea6a81 HTTP/1.1" 200 None 2025-09-28 04:48:03,351 - lava_callback - INFO - Completed processing callback for node 68d8b5e8a178a5bbc0ea6a81 INFO: 172.21.0.1:34538 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:48:06,653 - lava_callback - INFO - Received callback for node 68d8b5daa178a5bbc0ea6a1e INFO: 172.21.0.1:34542 - "POST /node/68d8b5daa178a5bbc0ea6a1e HTTP/1.0" 202 Accepted 2025-09-28 04:48:06,659 - lava_callback - INFO - Starting async job submit for node 68d8b5daa178a5bbc0ea6a1e 2025-09-28 04:48:06,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:06,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5daa178a5bbc0ea6a1e HTTP/1.1" 200 None 2025-09-28 04:48:06,691 - lava_callback - DEBUG - Processing node 68d8b5daa178a5bbc0ea6a1e: baseline-arm64 2025-09-28 04:48:06,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:48:07,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:48:07,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5daa178a5bbc0ea6a1e/log.txt.gz 2025-09-28 04:48:07,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:48:07,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:48:07,205 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5daa178a5bbc0ea6a1e/lava_callback.json.gz 2025-09-28 04:48:07,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:07,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5daa178a5bbc0ea6a1e HTTP/1.1" 200 None 2025-09-28 04:48:07,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:07,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:48:07,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:48:07,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5daa178a5bbc0ea6a1e HTTP/1.1" 200 None 2025-09-28 04:48:07,310 - lava_callback - INFO - Completed processing callback for node 68d8b5daa178a5bbc0ea6a1e INFO: 172.21.0.1:34550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37154 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:49:08,784 - lava_callback - INFO - Received callback for node 68d8bad9a178a5bbc0ea6c72 INFO: 172.21.0.1:37156 - "POST /node/68d8bad9a178a5bbc0ea6c72 HTTP/1.0" 202 Accepted 2025-09-28 04:49:08,793 - lava_callback - INFO - Starting async job submit for node 68d8bad9a178a5bbc0ea6c72 2025-09-28 04:49:08,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:08,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bad9a178a5bbc0ea6c72 HTTP/1.1" 200 None 2025-09-28 04:49:08,857 - lava_callback - DEBUG - Processing node 68d8bad9a178a5bbc0ea6c72: baseline-arm64 INFO: 172.21.0.1:37166 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:49:09,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:49:09,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:49:09,965 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bad9a178a5bbc0ea6c72/log.txt.gz 2025-09-28 04:49:09,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:49:10,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:49:10,166 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bad9a178a5bbc0ea6c72/lava_callback.json.gz DEBUG: Setup failed for 68d8bad9a178a5bbc0ea6c72. Transitting job node result: incomplete -> fail 2025-09-28 04:49:10,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:10,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bad9a178a5bbc0ea6c72 HTTP/1.1" 200 None 2025-09-28 04:49:10,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:10,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:49:10,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:10,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bad9a178a5bbc0ea6c72 HTTP/1.1" 200 None 2025-09-28 04:49:10,322 - lava_callback - INFO - Completed processing callback for node 68d8bad9a178a5bbc0ea6c72 INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55456 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:49:38,501 - lava_callback - INFO - Received callback for node 68d8b5dea178a5bbc0ea6a39 INFO: 172.21.0.1:55464 - "POST /node/68d8b5dea178a5bbc0ea6a39 HTTP/1.0" 202 Accepted 2025-09-28 04:49:38,510 - lava_callback - INFO - Starting async job submit for node 68d8b5dea178a5bbc0ea6a39 2025-09-28 04:49:38,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:38,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dea178a5bbc0ea6a39 HTTP/1.1" 200 None 2025-09-28 04:49:38,544 - lava_callback - DEBUG - Processing node 68d8b5dea178a5bbc0ea6a39: baseline-arm64 2025-09-28 04:49:38,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:49:38,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:49:38,899 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dea178a5bbc0ea6a39/log.txt.gz 2025-09-28 04:49:38,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:49:39,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:49:39,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8b5dea178a5bbc0ea6a39/lava_callback.json.gz 2025-09-28 04:49:39,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:39,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b5dea178a5bbc0ea6a39 HTTP/1.1" 200 None 2025-09-28 04:49:39,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:39,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:49:39,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:49:39,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b5dea178a5bbc0ea6a39 HTTP/1.1" 200 None 2025-09-28 04:49:39,098 - lava_callback - INFO - Completed processing callback for node 68d8b5dea178a5bbc0ea6a39 INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58632 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:50:13,098 - lava_callback - INFO - Received callback for node 68d61d0f30c58dac7d942359 INFO: 172.21.0.1:58638 - "POST /node/68d61d0f30c58dac7d942359 HTTP/1.0" 202 Accepted 2025-09-28 04:50:13,104 - lava_callback - INFO - Starting async job submit for node 68d61d0f30c58dac7d942359 2025-09-28 04:50:13,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:50:13,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d0f30c58dac7d942359 HTTP/1.1" 200 None 2025-09-28 04:50:13,138 - lava_callback - DEBUG - Processing node 68d61d0f30c58dac7d942359: baseline-arm-broonie 2025-09-28 04:50:13,370 - lava_callback - ERROR - Error processing callback for node 68d61d0f30c58dac7d942359: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:50:53,566 - lava_callback - INFO - Received callback for node 68d6179d30c58dac7d941fe4 INFO: 172.21.0.1:59324 - "POST /node/68d6179d30c58dac7d941fe4 HTTP/1.0" 202 Accepted 2025-09-28 04:50:53,574 - lava_callback - INFO - Starting async job submit for node 68d6179d30c58dac7d941fe4 2025-09-28 04:50:53,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:50:53,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179d30c58dac7d941fe4 HTTP/1.1" 200 None 2025-09-28 04:50:53,635 - lava_callback - DEBUG - Processing node 68d6179d30c58dac7d941fe4: ltp-syscalls-ipc 2025-09-28 04:50:54,521 - lava_callback - ERROR - Error processing callback for node 68d6179d30c58dac7d941fe4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49244 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:53:40,354 - lava_callback - INFO - Received callback for node 68d8bac1a178a5bbc0ea6c63 INFO: 172.21.0.1:44714 - "POST /node/68d8bac1a178a5bbc0ea6c63 HTTP/1.0" 202 Accepted 2025-09-28 04:53:40,360 - lava_callback - INFO - Starting async job submit for node 68d8bac1a178a5bbc0ea6c63 2025-09-28 04:53:40,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:40,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bac1a178a5bbc0ea6c63 HTTP/1.1" 200 None 2025-09-28 04:53:40,403 - lava_callback - DEBUG - Processing node 68d8bac1a178a5bbc0ea6c63: baseline-arm 2025-09-28 04:53:40,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:53:40,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:53:40,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bac1a178a5bbc0ea6c63/log.txt.gz 2025-09-28 04:53:40,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:53:40,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:53:40,999 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bac1a178a5bbc0ea6c63/lava_callback.json.gz DEBUG: Setup failed for 68d8bac1a178a5bbc0ea6c63. Transitting job node result: incomplete -> fail 2025-09-28 04:53:41,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:41,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bac1a178a5bbc0ea6c63 HTTP/1.1" 200 None 2025-09-28 04:53:41,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:41,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 04:53:41,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:41,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bac1a178a5bbc0ea6c63 HTTP/1.1" 200 None 2025-09-28 04:53:41,112 - lava_callback - INFO - Completed processing callback for node 68d8bac1a178a5bbc0ea6c63 2025-09-28 04:53:42,926 - lava_callback - INFO - Received callback for node 68d8bb1ea178a5bbc0ea6c97 INFO: 172.21.0.1:44730 - "POST /node/68d8bb1ea178a5bbc0ea6c97 HTTP/1.0" 202 Accepted 2025-09-28 04:53:42,934 - lava_callback - INFO - Starting async job submit for node 68d8bb1ea178a5bbc0ea6c97 2025-09-28 04:53:43,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:43,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bb1ea178a5bbc0ea6c97 HTTP/1.1" 200 None 2025-09-28 04:53:43,022 - lava_callback - DEBUG - Processing node 68d8bb1ea178a5bbc0ea6c97: ltp-smoketest 2025-09-28 04:53:43,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:53:43,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:53:43,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8bb1ea178a5bbc0ea6c97/log.txt.gz 2025-09-28 04:53:43,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:53:43,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:53:43,775 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoketest-68d8bb1ea178a5bbc0ea6c97/lava_callback.json.gz 2025-09-28 04:53:43,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:43,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bb1ea178a5bbc0ea6c97 HTTP/1.1" 200 None 2025-09-28 04:53:43,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:43,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae6a178a5bbc0ea5cb4 HTTP/1.1" 200 None 2025-09-28 04:53:43,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:53:43,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bb1ea178a5bbc0ea6c97 HTTP/1.1" 200 None 2025-09-28 04:53:43,945 - lava_callback - INFO - Completed processing callback for node 68d8bb1ea178a5bbc0ea6c97 INFO: 172.21.0.1:44732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:54:33,779 - lava_callback - INFO - Received callback for node 68d61d5930c58dac7d9423a8 INFO: 172.21.0.1:56114 - "POST /node/68d61d5930c58dac7d9423a8 HTTP/1.0" 202 Accepted 2025-09-28 04:54:33,785 - lava_callback - INFO - Starting async job submit for node 68d61d5930c58dac7d9423a8 2025-09-28 04:54:33,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:33,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d5930c58dac7d9423a8 HTTP/1.1" 200 None 2025-09-28 04:54:33,828 - lava_callback - DEBUG - Processing node 68d61d5930c58dac7d9423a8: ltp-cap-bounds 2025-09-28 04:54:34,157 - lava_callback - ERROR - Error processing callback for node 68d61d5930c58dac7d9423a8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56120 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:54:43,365 - lava_callback - INFO - Received callback for node 68d8bac8a178a5bbc0ea6c69 INFO: 172.21.0.1:59404 - "POST /node/68d8bac8a178a5bbc0ea6c69 HTTP/1.0" 202 Accepted 2025-09-28 04:54:43,371 - lava_callback - INFO - Starting async job submit for node 68d8bac8a178a5bbc0ea6c69 2025-09-28 04:54:43,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:43,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bac8a178a5bbc0ea6c69 HTTP/1.1" 200 None 2025-09-28 04:54:43,442 - lava_callback - DEBUG - Processing node 68d8bac8a178a5bbc0ea6c69: baseline-arm64 2025-09-28 04:54:43,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:54:44,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:54:44,087 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bac8a178a5bbc0ea6c69/log.txt.gz 2025-09-28 04:54:44,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:54:44,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:54:44,232 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bac8a178a5bbc0ea6c69/lava_callback.json.gz DEBUG: Setup failed for 68d8bac8a178a5bbc0ea6c69. Transitting job node result: incomplete -> fail 2025-09-28 04:54:44,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:44,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bac8a178a5bbc0ea6c69 HTTP/1.1" 200 None 2025-09-28 04:54:44,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:44,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:54:44,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:44,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bac8a178a5bbc0ea6c69 HTTP/1.1" 200 None 2025-09-28 04:54:44,382 - lava_callback - INFO - Completed processing callback for node 68d8bac8a178a5bbc0ea6c69 2025-09-28 04:54:44,763 - lava_callback - INFO - Received callback for node 68d8bac9a178a5bbc0ea6c6c INFO: 172.21.0.1:59410 - "POST /node/68d8bac9a178a5bbc0ea6c6c HTTP/1.0" 202 Accepted 2025-09-28 04:54:44,770 - lava_callback - INFO - Starting async job submit for node 68d8bac9a178a5bbc0ea6c6c 2025-09-28 04:54:44,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:44,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bac9a178a5bbc0ea6c6c HTTP/1.1" 200 None 2025-09-28 04:54:44,823 - lava_callback - DEBUG - Processing node 68d8bac9a178a5bbc0ea6c6c: baseline-arm64 2025-09-28 04:54:46,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:54:46,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:54:46,322 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bac9a178a5bbc0ea6c6c/log.txt.gz 2025-09-28 04:54:46,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:54:46,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:54:46,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bac9a178a5bbc0ea6c6c/lava_callback.json.gz INFO: 172.21.0.1:59416 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 68d8bac9a178a5bbc0ea6c6c. Transitting job node result: incomplete -> fail 2025-09-28 04:54:46,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:46,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bac9a178a5bbc0ea6c6c HTTP/1.1" 200 None 2025-09-28 04:54:46,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:46,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:54:46,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:54:46,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bac9a178a5bbc0ea6c6c HTTP/1.1" 200 None 2025-09-28 04:54:46,685 - lava_callback - INFO - Completed processing callback for node 68d8bac9a178a5bbc0ea6c6c INFO: 172.21.0.1:58502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48066 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:55:14,779 - lava_callback - INFO - Received callback for node 68d8badaa178a5bbc0ea6c75 INFO: 172.21.0.1:48074 - "POST /node/68d8badaa178a5bbc0ea6c75 HTTP/1.0" 202 Accepted 2025-09-28 04:55:14,787 - lava_callback - INFO - Starting async job submit for node 68d8badaa178a5bbc0ea6c75 2025-09-28 04:55:14,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:14,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8badaa178a5bbc0ea6c75 HTTP/1.1" 200 None 2025-09-28 04:55:14,840 - lava_callback - DEBUG - Processing node 68d8badaa178a5bbc0ea6c75: baseline-arm64 INFO: 172.21.0.1:48088 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 04:55:15,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:55:15,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:55:15,959 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8badaa178a5bbc0ea6c75/log.txt.gz 2025-09-28 04:55:15,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:55:16,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:55:16,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8badaa178a5bbc0ea6c75/lava_callback.json.gz DEBUG: Setup failed for 68d8badaa178a5bbc0ea6c75. Transitting job node result: incomplete -> fail 2025-09-28 04:55:16,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:16,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8badaa178a5bbc0ea6c75 HTTP/1.1" 200 None 2025-09-28 04:55:16,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:16,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:55:16,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:16,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8badaa178a5bbc0ea6c75 HTTP/1.1" 200 None 2025-09-28 04:55:16,282 - lava_callback - INFO - Completed processing callback for node 68d8badaa178a5bbc0ea6c75 2025-09-28 04:55:16,329 - lava_callback - INFO - Received callback for node 68d8badca178a5bbc0ea6c7b INFO: 172.21.0.1:48090 - "POST /node/68d8badca178a5bbc0ea6c7b HTTP/1.0" 202 Accepted 2025-09-28 04:55:16,342 - lava_callback - INFO - Starting async job submit for node 68d8badca178a5bbc0ea6c7b 2025-09-28 04:55:16,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:16,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8badca178a5bbc0ea6c7b HTTP/1.1" 200 None 2025-09-28 04:55:16,423 - lava_callback - DEBUG - Processing node 68d8badca178a5bbc0ea6c7b: baseline-arm64 INFO: 172.21.0.1:48098 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:55:17,771 - lava_callback - INFO - Received callback for node 68d8badda178a5bbc0ea6c7e INFO: 172.21.0.1:48110 - "POST /node/68d8badda178a5bbc0ea6c7e HTTP/1.0" 202 Accepted 2025-09-28 04:55:17,781 - lava_callback - INFO - Starting async job submit for node 68d8badda178a5bbc0ea6c7e 2025-09-28 04:55:17,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:17,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:55:17,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8badda178a5bbc0ea6c7e HTTP/1.1" 200 None 2025-09-28 04:55:17,853 - lava_callback - DEBUG - Processing node 68d8badda178a5bbc0ea6c7e: baseline-arm64 2025-09-28 04:55:18,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:55:18,071 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8badca178a5bbc0ea6c7b/log.txt.gz 2025-09-28 04:55:18,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:55:18,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:55:18,471 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8badca178a5bbc0ea6c7b/lava_callback.json.gz DEBUG: Setup failed for 68d8badca178a5bbc0ea6c7b. Transitting job node result: incomplete -> fail 2025-09-28 04:55:18,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:18,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8badca178a5bbc0ea6c7b HTTP/1.1" 200 None 2025-09-28 04:55:18,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:18,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:55:18,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:19,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8badca178a5bbc0ea6c7b HTTP/1.1" 200 None 2025-09-28 04:55:19,020 - lava_callback - INFO - Completed processing callback for node 68d8badca178a5bbc0ea6c7b 2025-09-28 04:55:19,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:55:19,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:55:19,511 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8badda178a5bbc0ea6c7e/log.txt.gz 2025-09-28 04:55:19,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:55:19,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:55:19,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8badda178a5bbc0ea6c7e/lava_callback.json.gz DEBUG: Setup failed for 68d8badda178a5bbc0ea6c7e. Transitting job node result: incomplete -> fail 2025-09-28 04:55:19,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:19,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8badda178a5bbc0ea6c7e HTTP/1.1" 200 None 2025-09-28 04:55:19,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:19,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:55:19,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:19,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8badda178a5bbc0ea6c7e HTTP/1.1" 200 None 2025-09-28 04:55:19,943 - lava_callback - INFO - Completed processing callback for node 68d8badda178a5bbc0ea6c7e INFO: 172.21.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:55:52,259 - lava_callback - INFO - Received callback for node 68d61cd230c58dac7d9422ab INFO: 172.21.0.1:46060 - "POST /node/68d61cd230c58dac7d9422ab HTTP/1.0" 202 Accepted 2025-09-28 04:55:52,264 - lava_callback - INFO - Starting async job submit for node 68d61cd230c58dac7d9422ab 2025-09-28 04:55:52,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:52,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd230c58dac7d9422ab HTTP/1.1" 200 None 2025-09-28 04:55:52,278 - lava_callback - DEBUG - Processing node 68d61cd230c58dac7d9422ab: baseline-arm 2025-09-28 04:55:52,278 - lava_callback - ERROR - Error processing callback for node 68d61cd230c58dac7d9422ab: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:55:52,894 - lava_callback - INFO - Received callback for node 68d61cd230c58dac7d9422ae INFO: 172.21.0.1:46074 - "POST /node/68d61cd230c58dac7d9422ae HTTP/1.0" 202 Accepted 2025-09-28 04:55:52,900 - lava_callback - INFO - Starting async job submit for node 68d61cd230c58dac7d9422ae 2025-09-28 04:55:52,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:52,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd230c58dac7d9422ae HTTP/1.1" 200 None 2025-09-28 04:55:52,913 - lava_callback - DEBUG - Processing node 68d61cd230c58dac7d9422ae: baseline-arm 2025-09-28 04:55:52,913 - lava_callback - ERROR - Error processing callback for node 68d61cd230c58dac7d9422ae: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:55:53,567 - lava_callback - INFO - Received callback for node 68d61cd430c58dac7d9422b1 INFO: 172.21.0.1:46078 - "POST /node/68d61cd430c58dac7d9422b1 HTTP/1.0" 202 Accepted 2025-09-28 04:55:53,572 - lava_callback - INFO - Starting async job submit for node 68d61cd430c58dac7d9422b1 2025-09-28 04:55:53,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd430c58dac7d9422b1 HTTP/1.1" 200 None 2025-09-28 04:55:53,585 - lava_callback - DEBUG - Processing node 68d61cd430c58dac7d9422b1: baseline-arm 2025-09-28 04:55:53,585 - lava_callback - ERROR - Error processing callback for node 68d61cd430c58dac7d9422b1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:55:54,242 - lava_callback - INFO - Received callback for node 68d61cd430c58dac7d9422b4 INFO: 172.21.0.1:46088 - "POST /node/68d61cd430c58dac7d9422b4 HTTP/1.0" 202 Accepted 2025-09-28 04:55:54,247 - lava_callback - INFO - Starting async job submit for node 68d61cd430c58dac7d9422b4 2025-09-28 04:55:54,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:54,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd430c58dac7d9422b4 HTTP/1.1" 200 None 2025-09-28 04:55:54,260 - lava_callback - DEBUG - Processing node 68d61cd430c58dac7d9422b4: baseline-arm 2025-09-28 04:55:54,261 - lava_callback - ERROR - Error processing callback for node 68d61cd430c58dac7d9422b4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:55:54,875 - lava_callback - INFO - Received callback for node 68d61cd630c58dac7d9422b7 INFO: 172.21.0.1:46090 - "POST /node/68d61cd630c58dac7d9422b7 HTTP/1.0" 202 Accepted 2025-09-28 04:55:54,880 - lava_callback - INFO - Starting async job submit for node 68d61cd630c58dac7d9422b7 2025-09-28 04:55:54,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:54,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd630c58dac7d9422b7 HTTP/1.1" 200 None 2025-09-28 04:55:54,901 - lava_callback - DEBUG - Processing node 68d61cd630c58dac7d9422b7: baseline-arm 2025-09-28 04:55:54,902 - lava_callback - ERROR - Error processing callback for node 68d61cd630c58dac7d9422b7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:55:55,556 - lava_callback - INFO - Received callback for node 68d61cd630c58dac7d9422ba INFO: 172.21.0.1:46104 - "POST /node/68d61cd630c58dac7d9422ba HTTP/1.0" 202 Accepted 2025-09-28 04:55:55,562 - lava_callback - INFO - Starting async job submit for node 68d61cd630c58dac7d9422ba 2025-09-28 04:55:55,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:55:55,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd630c58dac7d9422ba HTTP/1.1" 200 None 2025-09-28 04:55:55,575 - lava_callback - DEBUG - Processing node 68d61cd630c58dac7d9422ba: baseline-arm 2025-09-28 04:55:55,575 - lava_callback - ERROR - Error processing callback for node 68d61cd630c58dac7d9422ba: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47556 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:56:12,162 - lava_callback - INFO - Received callback for node 68d61cd830c58dac7d9422bd INFO: 172.21.0.1:47570 - "POST /node/68d61cd830c58dac7d9422bd HTTP/1.0" 202 Accepted 2025-09-28 04:56:12,168 - lava_callback - INFO - Starting async job submit for node 68d61cd830c58dac7d9422bd 2025-09-28 04:56:12,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:12,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd830c58dac7d9422bd HTTP/1.1" 200 None 2025-09-28 04:56:12,182 - lava_callback - DEBUG - Processing node 68d61cd830c58dac7d9422bd: baseline-arm 2025-09-28 04:56:12,183 - lava_callback - ERROR - Error processing callback for node 68d61cd830c58dac7d9422bd: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:12,837 - lava_callback - INFO - Received callback for node 68d61cd830c58dac7d9422c0 INFO: 172.21.0.1:47584 - "POST /node/68d61cd830c58dac7d9422c0 HTTP/1.0" 202 Accepted 2025-09-28 04:56:12,844 - lava_callback - INFO - Starting async job submit for node 68d61cd830c58dac7d9422c0 2025-09-28 04:56:12,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:12,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cd830c58dac7d9422c0 HTTP/1.1" 200 None 2025-09-28 04:56:12,857 - lava_callback - DEBUG - Processing node 68d61cd830c58dac7d9422c0: baseline-arm 2025-09-28 04:56:12,857 - lava_callback - ERROR - Error processing callback for node 68d61cd830c58dac7d9422c0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:56:31,851 - lava_callback - INFO - Received callback for node 68d61cf230c58dac7d94230b INFO: 172.21.0.1:36456 - "POST /node/68d61cf230c58dac7d94230b HTTP/1.0" 202 Accepted 2025-09-28 04:56:31,856 - lava_callback - INFO - Starting async job submit for node 68d61cf230c58dac7d94230b 2025-09-28 04:56:31,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:31,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cf230c58dac7d94230b HTTP/1.1" 200 None 2025-09-28 04:56:31,919 - lava_callback - DEBUG - Processing node 68d61cf230c58dac7d94230b: baseline-arm 2025-09-28 04:56:31,919 - lava_callback - ERROR - Error processing callback for node 68d61cf230c58dac7d94230b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:32,487 - lava_callback - INFO - Received callback for node 68d61cf330c58dac7d94230e INFO: 172.21.0.1:36462 - "POST /node/68d61cf330c58dac7d94230e HTTP/1.0" 202 Accepted 2025-09-28 04:56:32,492 - lava_callback - INFO - Starting async job submit for node 68d61cf330c58dac7d94230e 2025-09-28 04:56:32,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:32,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cf330c58dac7d94230e HTTP/1.1" 200 None 2025-09-28 04:56:32,507 - lava_callback - DEBUG - Processing node 68d61cf330c58dac7d94230e: baseline-arm 2025-09-28 04:56:32,507 - lava_callback - ERROR - Error processing callback for node 68d61cf330c58dac7d94230e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:33,134 - lava_callback - INFO - Received callback for node 68d61cf430c58dac7d942311 INFO: 172.21.0.1:36468 - "POST /node/68d61cf430c58dac7d942311 HTTP/1.0" 202 Accepted 2025-09-28 04:56:33,139 - lava_callback - INFO - Starting async job submit for node 68d61cf430c58dac7d942311 2025-09-28 04:56:33,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:33,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cf430c58dac7d942311 HTTP/1.1" 200 None 2025-09-28 04:56:33,275 - lava_callback - DEBUG - Processing node 68d61cf430c58dac7d942311: baseline-arm 2025-09-28 04:56:33,275 - lava_callback - ERROR - Error processing callback for node 68d61cf430c58dac7d942311: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:33,777 - lava_callback - INFO - Received callback for node 68d61cf530c58dac7d942314 INFO: 172.21.0.1:36476 - "POST /node/68d61cf530c58dac7d942314 HTTP/1.0" 202 Accepted 2025-09-28 04:56:33,782 - lava_callback - INFO - Starting async job submit for node 68d61cf530c58dac7d942314 2025-09-28 04:56:33,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:33,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cf530c58dac7d942314 HTTP/1.1" 200 None 2025-09-28 04:56:33,797 - lava_callback - DEBUG - Processing node 68d61cf530c58dac7d942314: baseline-arm 2025-09-28 04:56:33,797 - lava_callback - ERROR - Error processing callback for node 68d61cf530c58dac7d942314: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:34,405 - lava_callback - INFO - Received callback for node 68d61cf730c58dac7d942317 INFO: 172.21.0.1:36482 - "POST /node/68d61cf730c58dac7d942317 HTTP/1.0" 202 Accepted 2025-09-28 04:56:34,410 - lava_callback - INFO - Starting async job submit for node 68d61cf730c58dac7d942317 2025-09-28 04:56:34,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:34,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cf730c58dac7d942317 HTTP/1.1" 200 None 2025-09-28 04:56:34,423 - lava_callback - DEBUG - Processing node 68d61cf730c58dac7d942317: baseline-arm 2025-09-28 04:56:34,423 - lava_callback - ERROR - Error processing callback for node 68d61cf730c58dac7d942317: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:35,057 - lava_callback - INFO - Received callback for node 68d61cf830c58dac7d94231a INFO: 172.21.0.1:36492 - "POST /node/68d61cf830c58dac7d94231a HTTP/1.0" 202 Accepted 2025-09-28 04:56:35,062 - lava_callback - INFO - Starting async job submit for node 68d61cf830c58dac7d94231a 2025-09-28 04:56:35,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:35,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cf830c58dac7d94231a HTTP/1.1" 200 None 2025-09-28 04:56:35,076 - lava_callback - DEBUG - Processing node 68d61cf830c58dac7d94231a: baseline-arm 2025-09-28 04:56:35,076 - lava_callback - ERROR - Error processing callback for node 68d61cf830c58dac7d94231a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:35,671 - lava_callback - INFO - Received callback for node 68d61cfa30c58dac7d94231d INFO: 172.21.0.1:36496 - "POST /node/68d61cfa30c58dac7d94231d HTTP/1.0" 202 Accepted 2025-09-28 04:56:35,677 - lava_callback - INFO - Starting async job submit for node 68d61cfa30c58dac7d94231d 2025-09-28 04:56:35,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:35,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cfa30c58dac7d94231d HTTP/1.1" 200 None 2025-09-28 04:56:35,690 - lava_callback - DEBUG - Processing node 68d61cfa30c58dac7d94231d: baseline-arm 2025-09-28 04:56:35,691 - lava_callback - ERROR - Error processing callback for node 68d61cfa30c58dac7d94231d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:56:36,300 - lava_callback - INFO - Received callback for node 68d61cfa30c58dac7d942320 INFO: 172.21.0.1:36506 - "POST /node/68d61cfa30c58dac7d942320 HTTP/1.0" 202 Accepted 2025-09-28 04:56:36,305 - lava_callback - INFO - Starting async job submit for node 68d61cfa30c58dac7d942320 2025-09-28 04:56:36,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:36,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61cfa30c58dac7d942320 HTTP/1.1" 200 None 2025-09-28 04:56:36,318 - lava_callback - DEBUG - Processing node 68d61cfa30c58dac7d942320: baseline-arm 2025-09-28 04:56:36,318 - lava_callback - ERROR - Error processing callback for node 68d61cfa30c58dac7d942320: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52340 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:56:55,072 - lava_callback - INFO - Received callback for node 68d61d1530c58dac7d942362 INFO: 172.21.0.1:42936 - "POST /node/68d61d1530c58dac7d942362 HTTP/1.0" 202 Accepted 2025-09-28 04:56:55,078 - lava_callback - INFO - Starting async job submit for node 68d61d1530c58dac7d942362 2025-09-28 04:56:55,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:56:55,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d1530c58dac7d942362 HTTP/1.1" 200 None 2025-09-28 04:56:55,092 - lava_callback - DEBUG - Processing node 68d61d1530c58dac7d942362: baseline-arm-broonie 2025-09-28 04:56:55,092 - lava_callback - ERROR - Error processing callback for node 68d61d1530c58dac7d942362: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:57:04,859 - lava_callback - INFO - Received callback for node 68d61d1a30c58dac7d94236b INFO: 172.21.0.1:34324 - "POST /node/68d61d1a30c58dac7d94236b HTTP/1.0" 202 Accepted 2025-09-28 04:57:04,867 - lava_callback - INFO - Starting async job submit for node 68d61d1a30c58dac7d94236b 2025-09-28 04:57:04,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:04,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d1a30c58dac7d94236b HTTP/1.1" 200 None 2025-09-28 04:57:04,889 - lava_callback - DEBUG - Processing node 68d61d1a30c58dac7d94236b: baseline-arm-clabbe 2025-09-28 04:57:04,889 - lava_callback - ERROR - Error processing callback for node 68d61d1a30c58dac7d94236b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:57:05,549 - lava_callback - INFO - Received callback for node 68d61d1d30c58dac7d94236e INFO: 172.21.0.1:34332 - "POST /node/68d61d1d30c58dac7d94236e HTTP/1.0" 202 Accepted 2025-09-28 04:57:05,556 - lava_callback - INFO - Starting async job submit for node 68d61d1d30c58dac7d94236e 2025-09-28 04:57:05,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:05,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d1d30c58dac7d94236e HTTP/1.1" 200 None 2025-09-28 04:57:05,571 - lava_callback - DEBUG - Processing node 68d61d1d30c58dac7d94236e: baseline-arm-clabbe 2025-09-28 04:57:05,571 - lava_callback - ERROR - Error processing callback for node 68d61d1d30c58dac7d94236e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39558 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:57:14,620 - lava_callback - INFO - Received callback for node 68d6179c30c58dac7d941fe1 INFO: 172.21.0.1:39562 - "POST /node/68d6179c30c58dac7d941fe1 HTTP/1.0" 202 Accepted 2025-09-28 04:57:14,649 - lava_callback - INFO - Starting async job submit for node 68d6179c30c58dac7d941fe1 INFO: 172.21.0.1:39564 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 04:57:16,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:16,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6179c30c58dac7d941fe1 HTTP/1.1" 200 None 2025-09-28 04:57:16,334 - lava_callback - DEBUG - Processing node 68d6179c30c58dac7d941fe1: ltp-syscalls 2025-09-28 04:57:16,444 - lava_callback - INFO - Received callback for node 68d61d5b30c58dac7d9423ab INFO: 172.21.0.1:39566 - "POST /node/68d61d5b30c58dac7d9423ab HTTP/1.0" 202 Accepted 2025-09-28 04:57:16,475 - lava_callback - INFO - Starting async job submit for node 68d61d5b30c58dac7d9423ab INFO: 172.21.0.1:39570 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:57:16,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:16,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d5b30c58dac7d9423ab HTTP/1.1" 200 None 2025-09-28 04:57:16,798 - lava_callback - DEBUG - Processing node 68d61d5b30c58dac7d9423ab: ltp-containers 2025-09-28 04:57:18,935 - lava_callback - ERROR - Error processing callback for node 68d61d5b30c58dac7d9423ab: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-28 04:57:19,433 - lava_callback - INFO - Received callback for node 68d8bcd4a178a5bbc0ea6cb0 INFO: 172.21.0.1:40150 - "POST /node/68d8bcd4a178a5bbc0ea6cb0 HTTP/1.0" 202 Accepted 2025-09-28 04:57:19,483 - lava_callback - INFO - Starting async job submit for node 68d8bcd4a178a5bbc0ea6cb0 2025-09-28 04:57:19,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:19,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd4a178a5bbc0ea6cb0 HTTP/1.1" 200 None 2025-09-28 04:57:19,665 - lava_callback - DEBUG - Processing node 68d8bcd4a178a5bbc0ea6cb0: baseline-arm64 2025-09-28 04:57:22,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:57:23,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:57:23,138 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd4a178a5bbc0ea6cb0/log.txt.gz 2025-09-28 04:57:23,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:57:23,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:57:23,535 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd4a178a5bbc0ea6cb0/lava_callback.json.gz DEBUG: Setup failed for 68d8bcd4a178a5bbc0ea6cb0. Transitting job node result: incomplete -> fail 2025-09-28 04:57:23,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:23,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd4a178a5bbc0ea6cb0 HTTP/1.1" 200 None 2025-09-28 04:57:23,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:23,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:57:23,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:24,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bcd4a178a5bbc0ea6cb0 HTTP/1.1" 200 None 2025-09-28 04:57:24,217 - lava_callback - INFO - Completed processing callback for node 68d8bcd4a178a5bbc0ea6cb0 INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:57:27,844 - lava_callback - ERROR - Error processing callback for node 68d6179c30c58dac7d941fe1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:57:44,876 - lava_callback - INFO - Received callback for node 68d61d4330c58dac7d942388 INFO: 172.21.0.1:35002 - "POST /node/68d61d4330c58dac7d942388 HTTP/1.0" 202 Accepted 2025-09-28 04:57:44,881 - lava_callback - INFO - Starting async job submit for node 68d61d4330c58dac7d942388 2025-09-28 04:57:44,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:44,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d4330c58dac7d942388 HTTP/1.1" 200 None 2025-09-28 04:57:44,894 - lava_callback - DEBUG - Processing node 68d61d4330c58dac7d942388: baseline-arm-clabbe 2025-09-28 04:57:44,894 - lava_callback - ERROR - Error processing callback for node 68d61d4330c58dac7d942388: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:57:45,523 - lava_callback - INFO - Received callback for node 68d61d4630c58dac7d942390 INFO: 172.21.0.1:35006 - "POST /node/68d61d4630c58dac7d942390 HTTP/1.0" 202 Accepted 2025-09-28 04:57:45,528 - lava_callback - INFO - Starting async job submit for node 68d61d4630c58dac7d942390 2025-09-28 04:57:45,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:57:45,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d4630c58dac7d942390 HTTP/1.1" 200 None 2025-09-28 04:57:45,542 - lava_callback - DEBUG - Processing node 68d61d4630c58dac7d942390: baseline-arm-clabbe 2025-09-28 04:57:45,542 - lava_callback - ERROR - Error processing callback for node 68d61d4630c58dac7d942390: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:35020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33966 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:58:04,926 - lava_callback - INFO - Received callback for node 68d61d4e30c58dac7d94239c INFO: 172.21.0.1:36380 - "POST /node/68d61d4e30c58dac7d94239c HTTP/1.0" 202 Accepted 2025-09-28 04:58:04,932 - lava_callback - INFO - Starting async job submit for node 68d61d4e30c58dac7d94239c 2025-09-28 04:58:04,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:04,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d4e30c58dac7d94239c HTTP/1.1" 200 None 2025-09-28 04:58:04,954 - lava_callback - DEBUG - Processing node 68d61d4e30c58dac7d94239c: baseline-arm-clabbe 2025-09-28 04:58:04,954 - lava_callback - ERROR - Error processing callback for node 68d61d4e30c58dac7d94239c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:05,577 - lava_callback - INFO - Received callback for node 68d61d4830c58dac7d942393 INFO: 172.21.0.1:36396 - "POST /node/68d61d4830c58dac7d942393 HTTP/1.0" 202 Accepted 2025-09-28 04:58:05,582 - lava_callback - INFO - Starting async job submit for node 68d61d4830c58dac7d942393 2025-09-28 04:58:05,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:05,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d4830c58dac7d942393 HTTP/1.1" 200 None 2025-09-28 04:58:05,595 - lava_callback - DEBUG - Processing node 68d61d4830c58dac7d942393: baseline-arm-clabbe 2025-09-28 04:58:05,595 - lava_callback - ERROR - Error processing callback for node 68d61d4830c58dac7d942393: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 04:58:15,246 - lava_callback - INFO - Received callback for node 68d61d6130c58dac7d9423b7 INFO: 172.21.0.1:33762 - "POST /node/68d61d6130c58dac7d9423b7 HTTP/1.0" 202 Accepted 2025-09-28 04:58:15,251 - lava_callback - INFO - Starting async job submit for node 68d61d6130c58dac7d9423b7 2025-09-28 04:58:15,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:15,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6130c58dac7d9423b7 HTTP/1.1" 200 None 2025-09-28 04:58:15,268 - lava_callback - DEBUG - Processing node 68d61d6130c58dac7d9423b7: ltp-fs-bind 2025-09-28 04:58:15,268 - lava_callback - ERROR - Error processing callback for node 68d61d6130c58dac7d9423b7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:16,000 - lava_callback - INFO - Received callback for node 68d61d6330c58dac7d9423ba INFO: 172.21.0.1:33768 - "POST /node/68d61d6330c58dac7d9423ba HTTP/1.0" 202 Accepted 2025-09-28 04:58:16,006 - lava_callback - INFO - Starting async job submit for node 68d61d6330c58dac7d9423ba 2025-09-28 04:58:16,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:16,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6330c58dac7d9423ba HTTP/1.1" 200 None 2025-09-28 04:58:16,024 - lava_callback - DEBUG - Processing node 68d61d6330c58dac7d9423ba: ltp-fsx 2025-09-28 04:58:16,024 - lava_callback - ERROR - Error processing callback for node 68d61d6330c58dac7d9423ba: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:58:16,751 - lava_callback - INFO - Received callback for node 68d61d6530c58dac7d9423bd INFO: 172.21.0.1:33776 - "POST /node/68d61d6530c58dac7d9423bd HTTP/1.0" 202 Accepted 2025-09-28 04:58:16,756 - lava_callback - INFO - Starting async job submit for node 68d61d6530c58dac7d9423bd 2025-09-28 04:58:16,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:16,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6530c58dac7d9423bd HTTP/1.1" 200 None 2025-09-28 04:58:16,770 - lava_callback - DEBUG - Processing node 68d61d6530c58dac7d9423bd: ltp-input 2025-09-28 04:58:16,770 - lava_callback - ERROR - Error processing callback for node 68d61d6530c58dac7d9423bd: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:17,491 - lava_callback - INFO - Received callback for node 68d61d6630c58dac7d9423c0 INFO: 172.21.0.1:33782 - "POST /node/68d61d6630c58dac7d9423c0 HTTP/1.0" 202 Accepted 2025-09-28 04:58:17,497 - lava_callback - INFO - Starting async job submit for node 68d61d6630c58dac7d9423c0 2025-09-28 04:58:17,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:17,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6630c58dac7d9423c0 HTTP/1.1" 200 None 2025-09-28 04:58:17,510 - lava_callback - DEBUG - Processing node 68d61d6630c58dac7d9423c0: ltp-ipc 2025-09-28 04:58:17,510 - lava_callback - ERROR - Error processing callback for node 68d61d6630c58dac7d9423c0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:18,205 - lava_callback - INFO - Received callback for node 68d61d6830c58dac7d9423c3 INFO: 172.21.0.1:33798 - "POST /node/68d61d6830c58dac7d9423c3 HTTP/1.0" 202 Accepted 2025-09-28 04:58:18,211 - lava_callback - INFO - Starting async job submit for node 68d61d6830c58dac7d9423c3 2025-09-28 04:58:18,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:18,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6830c58dac7d9423c3 HTTP/1.1" 200 None 2025-09-28 04:58:18,224 - lava_callback - DEBUG - Processing node 68d61d6830c58dac7d9423c3: ltp-pty 2025-09-28 04:58:18,224 - lava_callback - ERROR - Error processing callback for node 68d61d6830c58dac7d9423c3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48784 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:58:35,211 - lava_callback - INFO - Received callback for node 68d61d6a30c58dac7d9423c6 INFO: 172.21.0.1:51984 - "POST /node/68d61d6a30c58dac7d9423c6 HTTP/1.0" 202 Accepted 2025-09-28 04:58:35,217 - lava_callback - INFO - Starting async job submit for node 68d61d6a30c58dac7d9423c6 2025-09-28 04:58:35,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:35,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6a30c58dac7d9423c6 HTTP/1.1" 200 None 2025-09-28 04:58:35,252 - lava_callback - DEBUG - Processing node 68d61d6a30c58dac7d9423c6: ltp-sched 2025-09-28 04:58:35,252 - lava_callback - ERROR - Error processing callback for node 68d61d6a30c58dac7d9423c6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:35,936 - lava_callback - INFO - Received callback for node 68d61d6c30c58dac7d9423c9 INFO: 172.21.0.1:51998 - "POST /node/68d61d6c30c58dac7d9423c9 HTTP/1.0" 202 Accepted 2025-09-28 04:58:35,942 - lava_callback - INFO - Starting async job submit for node 68d61d6c30c58dac7d9423c9 2025-09-28 04:58:35,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:35,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6c30c58dac7d9423c9 HTTP/1.1" 200 None 2025-09-28 04:58:35,955 - lava_callback - DEBUG - Processing node 68d61d6c30c58dac7d9423c9: ltp-smoketest 2025-09-28 04:58:35,955 - lava_callback - ERROR - Error processing callback for node 68d61d6c30c58dac7d9423c9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52006 - "GET / HTTP/1.0" 200 OK 2025-09-28 04:58:36,671 - lava_callback - INFO - Received callback for node 68d61d6f30c58dac7d9423cf INFO: 172.21.0.1:52008 - "POST /node/68d61d6f30c58dac7d9423cf HTTP/1.0" 202 Accepted 2025-09-28 04:58:36,677 - lava_callback - INFO - Starting async job submit for node 68d61d6f30c58dac7d9423cf 2025-09-28 04:58:36,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:36,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6f30c58dac7d9423cf HTTP/1.1" 200 None 2025-09-28 04:58:36,692 - lava_callback - DEBUG - Processing node 68d61d6f30c58dac7d9423cf: ltp-syscalls-ipc 2025-09-28 04:58:36,692 - lava_callback - ERROR - Error processing callback for node 68d61d6f30c58dac7d9423cf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:37,389 - lava_callback - INFO - Received callback for node 68d61d7030c58dac7d9423d2 INFO: 172.21.0.1:52018 - "POST /node/68d61d7030c58dac7d9423d2 HTTP/1.0" 202 Accepted 2025-09-28 04:58:37,394 - lava_callback - INFO - Starting async job submit for node 68d61d7030c58dac7d9423d2 2025-09-28 04:58:37,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:37,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d7030c58dac7d9423d2 HTTP/1.1" 200 None 2025-09-28 04:58:37,417 - lava_callback - DEBUG - Processing node 68d61d7030c58dac7d9423d2: ltp-timers 2025-09-28 04:58:37,417 - lava_callback - ERROR - Error processing callback for node 68d61d7030c58dac7d9423d2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:38,148 - lava_callback - INFO - Received callback for node 68d61d7230c58dac7d9423d5 INFO: 172.21.0.1:52028 - "POST /node/68d61d7230c58dac7d9423d5 HTTP/1.0" 202 Accepted 2025-09-28 04:58:38,154 - lava_callback - INFO - Starting async job submit for node 68d61d7230c58dac7d9423d5 2025-09-28 04:58:38,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:38,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d7230c58dac7d9423d5 HTTP/1.1" 200 None 2025-09-28 04:58:38,179 - lava_callback - DEBUG - Processing node 68d61d7230c58dac7d9423d5: ltp-watchqueue 2025-09-28 04:58:38,179 - lava_callback - ERROR - Error processing callback for node 68d61d7230c58dac7d9423d5: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 04:58:39,295 - lava_callback - INFO - Received callback for node 68d8bcd8a178a5bbc0ea6cb9 INFO: 172.21.0.1:52032 - "POST /node/68d8bcd8a178a5bbc0ea6cb9 HTTP/1.0" 202 Accepted 2025-09-28 04:58:39,302 - lava_callback - INFO - Starting async job submit for node 68d8bcd8a178a5bbc0ea6cb9 2025-09-28 04:58:39,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:39,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd8a178a5bbc0ea6cb9 HTTP/1.1" 200 None 2025-09-28 04:58:39,356 - lava_callback - DEBUG - Processing node 68d8bcd8a178a5bbc0ea6cb9: baseline-arm64 2025-09-28 04:58:40,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:58:40,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:58:40,557 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd8a178a5bbc0ea6cb9/log.txt.gz 2025-09-28 04:58:40,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 04:58:40,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 04:58:40,751 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd8a178a5bbc0ea6cb9/lava_callback.json.gz DEBUG: Setup failed for 68d8bcd8a178a5bbc0ea6cb9. Transitting job node result: incomplete -> fail 2025-09-28 04:58:40,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:40,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd8a178a5bbc0ea6cb9 HTTP/1.1" 200 None 2025-09-28 04:58:40,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:40,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 04:58:40,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 04:58:40,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bcd8a178a5bbc0ea6cb9 HTTP/1.1" 200 None 2025-09-28 04:58:40,894 - lava_callback - INFO - Completed processing callback for node 68d8bcd8a178a5bbc0ea6cb9 INFO: 172.21.0.1:46762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51898 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:01:44,414 - lava_callback - INFO - Received callback for node 68d7fe698dd6b39fd1aeb869 INFO: 172.21.0.1:37544 - "POST /node/68d7fe698dd6b39fd1aeb869 HTTP/1.0" 202 Accepted 2025-09-28 05:01:44,421 - lava_callback - INFO - Starting async job submit for node 68d7fe698dd6b39fd1aeb869 2025-09-28 05:01:44,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:01:44,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe698dd6b39fd1aeb869 HTTP/1.1" 200 None 2025-09-28 05:01:44,475 - lava_callback - DEBUG - Processing node 68d7fe698dd6b39fd1aeb869: baseline-arm64 2025-09-28 05:01:45,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:01:45,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:01:45,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe698dd6b39fd1aeb869/log.txt.gz 2025-09-28 05:01:45,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:01:45,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:01:45,709 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe698dd6b39fd1aeb869/lava_callback.json.gz DEBUG: Setup failed for 68d7fe698dd6b39fd1aeb869. Transitting job node result: incomplete -> fail 2025-09-28 05:01:45,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:01:45,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe698dd6b39fd1aeb869 HTTP/1.1" 200 None 2025-09-28 05:01:45,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:01:45,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:01:45,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:01:45,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fe698dd6b39fd1aeb869 HTTP/1.1" 200 None 2025-09-28 05:01:45,851 - lava_callback - INFO - Completed processing callback for node 68d7fe698dd6b39fd1aeb869 INFO: 172.21.0.1:37548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49182 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:03:18,975 - lava_callback - INFO - Received callback for node 68d8b174a178a5bbc0ea631b INFO: 172.21.0.1:49188 - "POST /node/68d8b174a178a5bbc0ea631b HTTP/1.0" 202 Accepted 2025-09-28 05:03:19,023 - lava_callback - INFO - Starting async job submit for node 68d8b174a178a5bbc0ea631b 2025-09-28 05:03:19,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:19,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b174a178a5bbc0ea631b HTTP/1.1" 200 None 2025-09-28 05:03:19,072 - lava_callback - DEBUG - Processing node 68d8b174a178a5bbc0ea631b: baseline-x86 2025-09-28 05:03:19,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:03:19,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:03:19,325 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b174a178a5bbc0ea631b/log.txt.gz 2025-09-28 05:03:19,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:03:19,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:03:19,492 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b174a178a5bbc0ea631b/lava_callback.json.gz 2025-09-28 05:03:19,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:19,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b174a178a5bbc0ea631b HTTP/1.1" 200 None 2025-09-28 05:03:19,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:19,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aafca178a5bbc0ea5cd2 HTTP/1.1" 200 None 2025-09-28 05:03:19,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:19,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b174a178a5bbc0ea631b HTTP/1.1" 200 None 2025-09-28 05:03:19,566 - lava_callback - INFO - Completed processing callback for node 68d8b174a178a5bbc0ea631b 2025-09-28 05:03:21,201 - lava_callback - INFO - Received callback for node 68d8af28a178a5bbc0ea62b3 INFO: 172.21.0.1:49214 - "POST /node/68d8af28a178a5bbc0ea62b3 HTTP/1.0" 202 Accepted 2025-09-28 05:03:21,211 - lava_callback - INFO - Starting async job submit for node 68d8af28a178a5bbc0ea62b3 2025-09-28 05:03:21,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:21,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8af28a178a5bbc0ea62b3 HTTP/1.1" 200 None 2025-09-28 05:03:21,266 - lava_callback - DEBUG - Processing node 68d8af28a178a5bbc0ea62b3: baseline-arm 2025-09-28 05:03:21,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:03:21,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:03:21,784 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8af28a178a5bbc0ea62b3/log.txt.gz 2025-09-28 05:03:21,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:03:21,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:03:21,935 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8af28a178a5bbc0ea62b3/lava_callback.json.gz DEBUG: Setup failed for 68d8af28a178a5bbc0ea62b3. Transitting job node result: incomplete -> fail 2025-09-28 05:03:22,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:22,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8af28a178a5bbc0ea62b3 HTTP/1.1" 200 None 2025-09-28 05:03:22,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:22,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 05:03:22,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:22,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8af28a178a5bbc0ea62b3 HTTP/1.1" 200 None 2025-09-28 05:03:22,075 - lava_callback - INFO - Completed processing callback for node 68d8af28a178a5bbc0ea62b3 INFO: 172.21.0.1:49230 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:03:32,219 - lava_callback - INFO - Received callback for node 68d8bcd5a178a5bbc0ea6cb3 INFO: 172.21.0.1:56142 - "POST /node/68d8bcd5a178a5bbc0ea6cb3 HTTP/1.0" 202 Accepted 2025-09-28 05:03:32,227 - lava_callback - INFO - Starting async job submit for node 68d8bcd5a178a5bbc0ea6cb3 2025-09-28 05:03:32,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:32,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd5a178a5bbc0ea6cb3 HTTP/1.1" 200 None 2025-09-28 05:03:32,279 - lava_callback - DEBUG - Processing node 68d8bcd5a178a5bbc0ea6cb3: baseline-arm64 2025-09-28 05:03:33,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:03:33,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:03:33,523 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd5a178a5bbc0ea6cb3/log.txt.gz 2025-09-28 05:03:33,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:03:33,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:03:33,723 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd5a178a5bbc0ea6cb3/lava_callback.json.gz DEBUG: Setup failed for 68d8bcd5a178a5bbc0ea6cb3. Transitting job node result: incomplete -> fail 2025-09-28 05:03:33,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:33,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd5a178a5bbc0ea6cb3 HTTP/1.1" 200 None 2025-09-28 05:03:33,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:33,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:03:33,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:03:33,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bcd5a178a5bbc0ea6cb3 HTTP/1.1" 200 None 2025-09-28 05:03:33,874 - lava_callback - INFO - Completed processing callback for node 68d8bcd5a178a5bbc0ea6cb3 INFO: 172.21.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53214 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:04:38,658 - lava_callback - INFO - Received callback for node 68d61ee330c58dac7d9424eb INFO: 172.21.0.1:53218 - "POST /node/68d61ee330c58dac7d9424eb HTTP/1.0" 202 Accepted 2025-09-28 05:04:38,663 - lava_callback - INFO - Starting async job submit for node 68d61ee330c58dac7d9424eb 2025-09-28 05:04:38,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:04:38,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61ee330c58dac7d9424eb HTTP/1.1" 200 None 2025-09-28 05:04:38,677 - lava_callback - DEBUG - Processing node 68d61ee330c58dac7d9424eb: baseline-arm64-baylibre 2025-09-28 05:04:38,677 - lava_callback - ERROR - Error processing callback for node 68d61ee330c58dac7d9424eb: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56020 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:05:18,697 - lava_callback - INFO - Received callback for node 68d61efc30c58dac7d94250e INFO: 172.21.0.1:56028 - "POST /node/68d61efc30c58dac7d94250e HTTP/1.0" 202 Accepted 2025-09-28 05:05:18,703 - lava_callback - INFO - Starting async job submit for node 68d61efc30c58dac7d94250e 2025-09-28 05:05:18,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:05:18,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61efc30c58dac7d94250e HTTP/1.1" 200 None 2025-09-28 05:05:18,716 - lava_callback - DEBUG - Processing node 68d61efc30c58dac7d94250e: baseline-arm64-baylibre 2025-09-28 05:05:18,716 - lava_callback - ERROR - Error processing callback for node 68d61efc30c58dac7d94250e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43506 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:05:45,260 - lava_callback - INFO - Received callback for node 68d61f2630c58dac7d942538 INFO: 172.21.0.1:51164 - "POST /node/68d61f2630c58dac7d942538 HTTP/1.0" 202 Accepted 2025-09-28 05:05:45,266 - lava_callback - INFO - Starting async job submit for node 68d61f2630c58dac7d942538 2025-09-28 05:05:45,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:05:45,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61f2630c58dac7d942538 HTTP/1.1" 200 None 2025-09-28 05:05:45,279 - lava_callback - DEBUG - Processing node 68d61f2630c58dac7d942538: baseline-arm64-clabbe 2025-09-28 05:05:45,279 - lava_callback - ERROR - Error processing callback for node 68d61f2630c58dac7d942538: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:51172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46692 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:06:03,724 - lava_callback - INFO - Received callback for node 68d8bdc8a178a5bbc0ea6cda INFO: 172.21.0.1:52950 - "POST /node/68d8bdc8a178a5bbc0ea6cda HTTP/1.0" 202 Accepted 2025-09-28 05:06:03,731 - lava_callback - INFO - Starting async job submit for node 68d8bdc8a178a5bbc0ea6cda 2025-09-28 05:06:03,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:03,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bdc8a178a5bbc0ea6cda HTTP/1.1" 200 None 2025-09-28 05:06:03,772 - lava_callback - DEBUG - Processing node 68d8bdc8a178a5bbc0ea6cda: ltp-syscalls-ipc 2025-09-28 05:06:04,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:06:04,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:06:04,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8bdc8a178a5bbc0ea6cda/log.txt.gz 2025-09-28 05:06:04,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:06:04,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:06:04,866 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-68d8bdc8a178a5bbc0ea6cda/lava_callback.json.gz 2025-09-28 05:06:04,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:04,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bdc8a178a5bbc0ea6cda HTTP/1.1" 200 None 2025-09-28 05:06:04,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:04,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 05:06:04,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:04,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bdc8a178a5bbc0ea6cda HTTP/1.1" 200 None 2025-09-28 05:06:04,978 - lava_callback - INFO - Completed processing callback for node 68d8bdc8a178a5bbc0ea6cda 2025-09-28 05:06:05,256 - lava_callback - INFO - Received callback for node 68d61f2f30c58dac7d942541 INFO: 172.21.0.1:52966 - "POST /node/68d61f2f30c58dac7d942541 HTTP/1.0" 202 Accepted 2025-09-28 05:06:05,262 - lava_callback - INFO - Starting async job submit for node 68d61f2f30c58dac7d942541 2025-09-28 05:06:05,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:05,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61f2f30c58dac7d942541 HTTP/1.1" 200 None 2025-09-28 05:06:05,275 - lava_callback - DEBUG - Processing node 68d61f2f30c58dac7d942541: baseline-arm64-clabbe 2025-09-28 05:06:05,275 - lava_callback - ERROR - Error processing callback for node 68d61f2f30c58dac7d942541: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 05:06:05,950 - lava_callback - INFO - Received callback for node 68d61f3230c58dac7d942544 INFO: 172.21.0.1:52968 - "POST /node/68d61f3230c58dac7d942544 HTTP/1.0" 202 Accepted 2025-09-28 05:06:05,956 - lava_callback - INFO - Starting async job submit for node 68d61f3230c58dac7d942544 2025-09-28 05:06:05,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:05,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61f3230c58dac7d942544 HTTP/1.1" 200 None 2025-09-28 05:06:05,971 - lava_callback - DEBUG - Processing node 68d61f3230c58dac7d942544: baseline-arm64-clabbe 2025-09-28 05:06:05,971 - lava_callback - ERROR - Error processing callback for node 68d61f3230c58dac7d942544: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52982 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:06:06,652 - lava_callback - INFO - Received callback for node 68d61f3730c58dac7d942547 INFO: 172.21.0.1:52996 - "POST /node/68d61f3730c58dac7d942547 HTTP/1.0" 202 Accepted 2025-09-28 05:06:06,657 - lava_callback - INFO - Starting async job submit for node 68d61f3730c58dac7d942547 2025-09-28 05:06:06,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:06,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61f3730c58dac7d942547 HTTP/1.1" 200 None 2025-09-28 05:06:06,671 - lava_callback - DEBUG - Processing node 68d61f3730c58dac7d942547: baseline-arm64-clabbe 2025-09-28 05:06:06,671 - lava_callback - ERROR - Error processing callback for node 68d61f3730c58dac7d942547: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 05:06:07,314 - lava_callback - INFO - Received callback for node 68d61f3a30c58dac7d94254a INFO: 172.21.0.1:53010 - "POST /node/68d61f3a30c58dac7d94254a HTTP/1.0" 202 Accepted 2025-09-28 05:06:07,319 - lava_callback - INFO - Starting async job submit for node 68d61f3a30c58dac7d94254a 2025-09-28 05:06:07,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:07,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61f3a30c58dac7d94254a HTTP/1.1" 200 None 2025-09-28 05:06:07,334 - lava_callback - DEBUG - Processing node 68d61f3a30c58dac7d94254a: baseline-arm64-clabbe 2025-09-28 05:06:07,334 - lava_callback - ERROR - Error processing callback for node 68d61f3a30c58dac7d94254a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:42426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:06:25,274 - lava_callback - INFO - Received callback for node 68d61f3d30c58dac7d94254d INFO: 172.21.0.1:38944 - "POST /node/68d61f3d30c58dac7d94254d HTTP/1.0" 202 Accepted 2025-09-28 05:06:25,280 - lava_callback - INFO - Starting async job submit for node 68d61f3d30c58dac7d94254d 2025-09-28 05:06:25,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:06:25,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61f3d30c58dac7d94254d HTTP/1.1" 200 None 2025-09-28 05:06:25,294 - lava_callback - DEBUG - Processing node 68d61f3d30c58dac7d94254d: baseline-arm64-clabbe 2025-09-28 05:06:25,294 - lava_callback - ERROR - Error processing callback for node 68d61f3d30c58dac7d94254d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58080 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:07:12,327 - lava_callback - INFO - Received callback for node 68d7fe6a8dd6b39fd1aeb86c INFO: 172.21.0.1:58094 - "POST /node/68d7fe6a8dd6b39fd1aeb86c HTTP/1.0" 202 Accepted 2025-09-28 05:07:12,334 - lava_callback - INFO - Starting async job submit for node 68d7fe6a8dd6b39fd1aeb86c 2025-09-28 05:07:12,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:12,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe6a8dd6b39fd1aeb86c HTTP/1.1" 200 None 2025-09-28 05:07:12,386 - lava_callback - DEBUG - Processing node 68d7fe6a8dd6b39fd1aeb86c: baseline-arm64 2025-09-28 05:07:13,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:07:13,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:07:13,389 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe6a8dd6b39fd1aeb86c/log.txt.gz 2025-09-28 05:07:13,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:07:13,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:07:13,606 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe6a8dd6b39fd1aeb86c/lava_callback.json.gz DEBUG: Setup failed for 68d7fe6a8dd6b39fd1aeb86c. Transitting job node result: incomplete -> fail 2025-09-28 05:07:13,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:13,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe6a8dd6b39fd1aeb86c HTTP/1.1" 200 None 2025-09-28 05:07:13,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:13,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:07:13,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:13,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fe6a8dd6b39fd1aeb86c HTTP/1.1" 200 None 2025-09-28 05:07:13,763 - lava_callback - INFO - Completed processing callback for node 68d7fe6a8dd6b39fd1aeb86c INFO: 172.21.0.1:58100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:07:32,027 - lava_callback - INFO - Received callback for node 68d7fe728dd6b39fd1aeb872 INFO: 172.21.0.1:38058 - "POST /node/68d7fe728dd6b39fd1aeb872 HTTP/1.0" 202 Accepted 2025-09-28 05:07:32,034 - lava_callback - INFO - Starting async job submit for node 68d7fe728dd6b39fd1aeb872 2025-09-28 05:07:32,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:32,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe728dd6b39fd1aeb872 HTTP/1.1" 200 None 2025-09-28 05:07:32,087 - lava_callback - DEBUG - Processing node 68d7fe728dd6b39fd1aeb872: baseline-arm64 2025-09-28 05:07:33,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:07:33,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:07:33,125 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe728dd6b39fd1aeb872/log.txt.gz 2025-09-28 05:07:33,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:07:33,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:07:33,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe728dd6b39fd1aeb872/lava_callback.json.gz DEBUG: Setup failed for 68d7fe728dd6b39fd1aeb872. Transitting job node result: incomplete -> fail 2025-09-28 05:07:33,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:33,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe728dd6b39fd1aeb872 HTTP/1.1" 200 None 2025-09-28 05:07:33,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:33,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:07:33,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:07:33,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fe728dd6b39fd1aeb872 HTTP/1.1" 200 None 2025-09-28 05:07:33,435 - lava_callback - INFO - Completed processing callback for node 68d7fe728dd6b39fd1aeb872 INFO: 172.21.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37054 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:09:50,392 - lava_callback - INFO - Received callback for node 68d8be46a178a5bbc0ea6ce8 INFO: 172.21.0.1:39060 - "POST /node/68d8be46a178a5bbc0ea6ce8 HTTP/1.0" 202 Accepted 2025-09-28 05:09:50,400 - lava_callback - INFO - Starting async job submit for node 68d8be46a178a5bbc0ea6ce8 2025-09-28 05:09:50,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:09:50,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8be46a178a5bbc0ea6ce8 HTTP/1.1" 200 None 2025-09-28 05:09:50,472 - lava_callback - DEBUG - Processing node 68d8be46a178a5bbc0ea6ce8: baseline-arm64 2025-09-28 05:09:51,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:09:51,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:09:51,571 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8be46a178a5bbc0ea6ce8/log.txt.gz 2025-09-28 05:09:51,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:09:51,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:09:51,727 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8be46a178a5bbc0ea6ce8/lava_callback.json.gz DEBUG: Setup failed for 68d8be46a178a5bbc0ea6ce8. Transitting job node result: incomplete -> fail 2025-09-28 05:09:51,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:09:51,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8be46a178a5bbc0ea6ce8 HTTP/1.1" 200 None 2025-09-28 05:09:51,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:09:51,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:09:51,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:09:51,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8be46a178a5bbc0ea6ce8 HTTP/1.1" 200 None 2025-09-28 05:09:51,861 - lava_callback - INFO - Completed processing callback for node 68d8be46a178a5bbc0ea6ce8 INFO: 172.21.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41418 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:10:59,034 - lava_callback - INFO - Received callback for node 68d6205130c58dac7d942776 INFO: 172.21.0.1:41432 - "POST /node/68d6205130c58dac7d942776 HTTP/1.0" 202 Accepted 2025-09-28 05:10:59,039 - lava_callback - INFO - Starting async job submit for node 68d6205130c58dac7d942776 2025-09-28 05:10:59,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:10:59,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6205130c58dac7d942776 HTTP/1.1" 200 None 2025-09-28 05:10:59,072 - lava_callback - DEBUG - Processing node 68d6205130c58dac7d942776: baseline-arm64-baylibre 2025-09-28 05:10:59,072 - lava_callback - ERROR - Error processing callback for node 68d6205130c58dac7d942776: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33480 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:11:12,145 - lava_callback - INFO - Received callback for node 68d8bcd9a178a5bbc0ea6cbc INFO: 172.21.0.1:33486 - "POST /node/68d8bcd9a178a5bbc0ea6cbc HTTP/1.0" 202 Accepted 2025-09-28 05:11:12,152 - lava_callback - INFO - Starting async job submit for node 68d8bcd9a178a5bbc0ea6cbc 2025-09-28 05:11:12,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:12,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd9a178a5bbc0ea6cbc HTTP/1.1" 200 None 2025-09-28 05:11:12,205 - lava_callback - DEBUG - Processing node 68d8bcd9a178a5bbc0ea6cbc: baseline-arm64 2025-09-28 05:11:13,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:11:13,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:11:13,311 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd9a178a5bbc0ea6cbc/log.txt.gz 2025-09-28 05:11:13,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:11:13,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:11:13,528 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bcd9a178a5bbc0ea6cbc/lava_callback.json.gz DEBUG: Setup failed for 68d8bcd9a178a5bbc0ea6cbc. Transitting job node result: incomplete -> fail 2025-09-28 05:11:13,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:13,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcd9a178a5bbc0ea6cbc HTTP/1.1" 200 None 2025-09-28 05:11:13,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:13,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:11:13,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:13,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bcd9a178a5bbc0ea6cbc HTTP/1.1" 200 None 2025-09-28 05:11:13,658 - lava_callback - INFO - Completed processing callback for node 68d8bcd9a178a5bbc0ea6cbc INFO: 172.21.0.1:33498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33526 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:11:19,040 - lava_callback - INFO - Received callback for node 68d6206730c58dac7d94278e INFO: 172.21.0.1:33542 - "POST /node/68d6206730c58dac7d94278e HTTP/1.0" 202 Accepted 2025-09-28 05:11:19,047 - lava_callback - INFO - Starting async job submit for node 68d6206730c58dac7d94278e 2025-09-28 05:11:19,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:19,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6206730c58dac7d94278e HTTP/1.1" 200 None 2025-09-28 05:11:19,068 - lava_callback - DEBUG - Processing node 68d6206730c58dac7d94278e: baseline-arm64-baylibre 2025-09-28 05:11:19,068 - lava_callback - ERROR - Error processing callback for node 68d6206730c58dac7d94278e: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:54874 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:11:34,542 - lava_callback - INFO - Received callback for node 68d61d5c30c58dac7d9423ae INFO: 172.21.0.1:37658 - "POST /node/68d61d5c30c58dac7d9423ae HTTP/1.0" 202 Accepted 2025-09-28 05:11:34,553 - lava_callback - INFO - Starting async job submit for node 68d61d5c30c58dac7d9423ae 2025-09-28 05:11:34,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:34,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d5c30c58dac7d9423ae HTTP/1.1" 200 None 2025-09-28 05:11:34,923 - lava_callback - DEBUG - Processing node 68d61d5c30c58dac7d9423ae: ltp-controllers INFO: 172.21.0.1:37672 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:11:37,697 - lava_callback - ERROR - Error processing callback for node 68d61d5c30c58dac7d9423ae: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-28 05:11:39,000 - lava_callback - INFO - Received callback for node 68d8be47a178a5bbc0ea6ceb INFO: 172.21.0.1:37680 - "POST /node/68d8be47a178a5bbc0ea6ceb HTTP/1.0" 202 Accepted 2025-09-28 05:11:39,008 - lava_callback - INFO - Starting async job submit for node 68d8be47a178a5bbc0ea6ceb 2025-09-28 05:11:39,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:39,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8be47a178a5bbc0ea6ceb HTTP/1.1" 200 None 2025-09-28 05:11:39,057 - lava_callback - DEBUG - Processing node 68d8be47a178a5bbc0ea6ceb: baseline-arm64 2025-09-28 05:11:40,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:11:40,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:11:40,179 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8be47a178a5bbc0ea6ceb/log.txt.gz 2025-09-28 05:11:40,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:11:40,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:11:40,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8be47a178a5bbc0ea6ceb/lava_callback.json.gz DEBUG: Setup failed for 68d8be47a178a5bbc0ea6ceb. Transitting job node result: incomplete -> fail 2025-09-28 05:11:40,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:40,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8be47a178a5bbc0ea6ceb HTTP/1.1" 200 None 2025-09-28 05:11:40,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:40,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:11:40,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:40,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8be47a178a5bbc0ea6ceb HTTP/1.1" 200 None 2025-09-28 05:11:40,512 - lava_callback - INFO - Completed processing callback for node 68d8be47a178a5bbc0ea6ceb 2025-09-28 05:11:43,357 - lava_callback - INFO - Received callback for node 68d6337fc1bc82a30b37ad30 INFO: 172.21.0.1:38614 - "POST /node/68d6337fc1bc82a30b37ad30 HTTP/1.0" 202 Accepted 2025-09-28 05:11:43,364 - lava_callback - INFO - Starting async job submit for node 68d6337fc1bc82a30b37ad30 2025-09-28 05:11:43,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:11:43,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6337fc1bc82a30b37ad30 HTTP/1.1" 200 None 2025-09-28 05:11:43,433 - lava_callback - DEBUG - Processing node 68d6337fc1bc82a30b37ad30: baseline-arm-broonie 2025-09-28 05:11:43,749 - lava_callback - ERROR - Error processing callback for node 68d6337fc1bc82a30b37ad30: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36392 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:12:05,701 - lava_callback - INFO - Received callback for node 68d6209730c58dac7d9427be INFO: 172.21.0.1:46094 - "POST /node/68d6209730c58dac7d9427be HTTP/1.0" 202 Accepted 2025-09-28 05:12:05,706 - lava_callback - INFO - Starting async job submit for node 68d6209730c58dac7d9427be 2025-09-28 05:12:05,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:05,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6209730c58dac7d9427be HTTP/1.1" 200 None 2025-09-28 05:12:05,720 - lava_callback - DEBUG - Processing node 68d6209730c58dac7d9427be: baseline-arm64-clabbe 2025-09-28 05:12:05,720 - lava_callback - ERROR - Error processing callback for node 68d6209730c58dac7d9427be: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 05:12:06,354 - lava_callback - INFO - Received callback for node 68d6209a30c58dac7d9427c1 INFO: 172.21.0.1:46108 - "POST /node/68d6209a30c58dac7d9427c1 HTTP/1.0" 202 Accepted 2025-09-28 05:12:06,360 - lava_callback - INFO - Starting async job submit for node 68d6209a30c58dac7d9427c1 2025-09-28 05:12:06,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:06,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6209a30c58dac7d9427c1 HTTP/1.1" 200 None 2025-09-28 05:12:06,375 - lava_callback - DEBUG - Processing node 68d6209a30c58dac7d9427c1: baseline-arm64-clabbe 2025-09-28 05:12:06,375 - lava_callback - ERROR - Error processing callback for node 68d6209a30c58dac7d9427c1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46124 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:12:07,021 - lava_callback - INFO - Received callback for node 68d6209030c58dac7d9427b8 INFO: 172.21.0.1:46132 - "POST /node/68d6209030c58dac7d9427b8 HTTP/1.0" 202 Accepted 2025-09-28 05:12:07,026 - lava_callback - INFO - Starting async job submit for node 68d6209030c58dac7d9427b8 2025-09-28 05:12:07,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:07,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6209030c58dac7d9427b8 HTTP/1.1" 200 None 2025-09-28 05:12:07,039 - lava_callback - DEBUG - Processing node 68d6209030c58dac7d9427b8: baseline-arm64-clabbe 2025-09-28 05:12:07,040 - lava_callback - ERROR - Error processing callback for node 68d6209030c58dac7d9427b8: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 05:12:07,660 - lava_callback - INFO - Received callback for node 68d6209e30c58dac7d9427c4 INFO: 172.21.0.1:46144 - "POST /node/68d6209e30c58dac7d9427c4 HTTP/1.0" 202 Accepted 2025-09-28 05:12:07,665 - lava_callback - INFO - Starting async job submit for node 68d6209e30c58dac7d9427c4 2025-09-28 05:12:07,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:07,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6209e30c58dac7d9427c4 HTTP/1.1" 200 None 2025-09-28 05:12:07,679 - lava_callback - DEBUG - Processing node 68d6209e30c58dac7d9427c4: baseline-arm64-clabbe 2025-09-28 05:12:07,679 - lava_callback - ERROR - Error processing callback for node 68d6209e30c58dac7d9427c4: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 05:12:08,316 - lava_callback - INFO - Received callback for node 68d620a030c58dac7d9427c7 INFO: 172.21.0.1:46158 - "POST /node/68d620a030c58dac7d9427c7 HTTP/1.0" 202 Accepted 2025-09-28 05:12:08,322 - lava_callback - INFO - Starting async job submit for node 68d620a030c58dac7d9427c7 2025-09-28 05:12:08,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:08,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d620a030c58dac7d9427c7 HTTP/1.1" 200 None 2025-09-28 05:12:08,336 - lava_callback - DEBUG - Processing node 68d620a030c58dac7d9427c7: baseline-arm64-clabbe 2025-09-28 05:12:08,336 - lava_callback - ERROR - Error processing callback for node 68d620a030c58dac7d9427c7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 05:12:08,978 - lava_callback - INFO - Received callback for node 68d620a330c58dac7d9427ca INFO: 172.21.0.1:46172 - "POST /node/68d620a330c58dac7d9427ca HTTP/1.0" 202 Accepted 2025-09-28 05:12:08,983 - lava_callback - INFO - Starting async job submit for node 68d620a330c58dac7d9427ca 2025-09-28 05:12:08,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:08,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d620a330c58dac7d9427ca HTTP/1.1" 200 None 2025-09-28 05:12:08,997 - lava_callback - DEBUG - Processing node 68d620a330c58dac7d9427ca: baseline-arm64-clabbe 2025-09-28 05:12:08,997 - lava_callback - ERROR - Error processing callback for node 68d620a330c58dac7d9427ca: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58034 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:12:19,645 - lava_callback - INFO - Received callback for node 68d8bf55a178a5bbc0ea6cf7 INFO: 172.21.0.1:57576 - "POST /node/68d8bf55a178a5bbc0ea6cf7 HTTP/1.0" 202 Accepted 2025-09-28 05:12:19,651 - lava_callback - INFO - Starting async job submit for node 68d8bf55a178a5bbc0ea6cf7 2025-09-28 05:12:19,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:19,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf55a178a5bbc0ea6cf7 HTTP/1.1" 200 None 2025-09-28 05:12:19,714 - lava_callback - DEBUG - Processing node 68d8bf55a178a5bbc0ea6cf7: baseline-arm 2025-09-28 05:12:20,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:12:20,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:12:20,171 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bf55a178a5bbc0ea6cf7/log.txt.gz 2025-09-28 05:12:20,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:12:20,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:12:20,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bf55a178a5bbc0ea6cf7/lava_callback.json.gz DEBUG: Setup failed for 68d8bf55a178a5bbc0ea6cf7. Transitting job node result: incomplete -> fail 2025-09-28 05:12:20,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:20,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf55a178a5bbc0ea6cf7 HTTP/1.1" 200 None 2025-09-28 05:12:20,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:20,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aae0a178a5bbc0ea5cae HTTP/1.1" 200 None 2025-09-28 05:12:20,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:12:20,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bf55a178a5bbc0ea6cf7 HTTP/1.1" 200 None 2025-09-28 05:12:20,430 - lava_callback - INFO - Completed processing callback for node 68d8bf55a178a5bbc0ea6cf7 INFO: 172.21.0.1:57592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37272 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:15:04,316 - lava_callback - INFO - Received callback for node 68d8bf94a178a5bbc0ea6d02 INFO: 172.21.0.1:41170 - "POST /node/68d8bf94a178a5bbc0ea6d02 HTTP/1.0" 202 Accepted 2025-09-28 05:15:04,323 - lava_callback - INFO - Starting async job submit for node 68d8bf94a178a5bbc0ea6d02 2025-09-28 05:15:04,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:04,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf94a178a5bbc0ea6d02 HTTP/1.1" 200 None 2025-09-28 05:15:04,400 - lava_callback - DEBUG - Processing node 68d8bf94a178a5bbc0ea6d02: baseline-arm64 2025-09-28 05:15:05,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:15:05,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:15:05,624 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf94a178a5bbc0ea6d02/log.txt.gz 2025-09-28 05:15:05,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:15:05,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:15:05,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf94a178a5bbc0ea6d02/lava_callback.json.gz DEBUG: Setup failed for 68d8bf94a178a5bbc0ea6d02. Transitting job node result: incomplete -> fail 2025-09-28 05:15:05,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:05,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf94a178a5bbc0ea6d02 HTTP/1.1" 200 None 2025-09-28 05:15:05,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:05,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:15:05,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:05,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bf94a178a5bbc0ea6d02 HTTP/1.1" 200 None 2025-09-28 05:15:05,967 - lava_callback - INFO - Completed processing callback for node 68d8bf94a178a5bbc0ea6d02 INFO: 172.21.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56974 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:15:53,897 - lava_callback - INFO - Received callback for node 68d8bf97a178a5bbc0ea6d0b INFO: 172.21.0.1:33660 - "POST /node/68d8bf97a178a5bbc0ea6d0b HTTP/1.0" 202 Accepted 2025-09-28 05:15:53,904 - lava_callback - INFO - Starting async job submit for node 68d8bf97a178a5bbc0ea6d0b 2025-09-28 05:15:53,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:53,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf97a178a5bbc0ea6d0b HTTP/1.1" 200 None 2025-09-28 05:15:53,958 - lava_callback - DEBUG - Processing node 68d8bf97a178a5bbc0ea6d0b: baseline-arm64 2025-09-28 05:15:55,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:15:55,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:15:55,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf97a178a5bbc0ea6d0b/log.txt.gz 2025-09-28 05:15:55,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:15:55,283 - lava_callback - INFO - Received callback for node 68d8bf95a178a5bbc0ea6d05 INFO: 172.21.0.1:33662 - "POST /node/68d8bf95a178a5bbc0ea6d05 HTTP/1.0" 202 Accepted 2025-09-28 05:15:55,291 - lava_callback - INFO - Starting async job submit for node 68d8bf95a178a5bbc0ea6d05 2025-09-28 05:15:55,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:55,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf95a178a5bbc0ea6d05 HTTP/1.1" 200 None 2025-09-28 05:15:55,356 - lava_callback - DEBUG - Processing node 68d8bf95a178a5bbc0ea6d05: baseline-arm64 2025-09-28 05:15:55,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:15:55,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf97a178a5bbc0ea6d0b/lava_callback.json.gz DEBUG: Setup failed for 68d8bf97a178a5bbc0ea6d0b. Transitting job node result: incomplete -> fail 2025-09-28 05:15:55,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:55,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf97a178a5bbc0ea6d0b HTTP/1.1" 200 None 2025-09-28 05:15:55,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:55,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:15:55,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:56,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bf97a178a5bbc0ea6d0b HTTP/1.1" 200 None 2025-09-28 05:15:56,033 - lava_callback - INFO - Completed processing callback for node 68d8bf97a178a5bbc0ea6d0b INFO: 172.21.0.1:33676 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:15:56,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:15:57,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:15:57,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf95a178a5bbc0ea6d05/log.txt.gz 2025-09-28 05:15:57,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:15:57,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:15:57,220 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf95a178a5bbc0ea6d05/lava_callback.json.gz DEBUG: Setup failed for 68d8bf95a178a5bbc0ea6d05. Transitting job node result: incomplete -> fail 2025-09-28 05:15:57,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:57,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf95a178a5bbc0ea6d05 HTTP/1.1" 200 None 2025-09-28 05:15:57,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:57,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:15:57,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:15:57,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bf95a178a5bbc0ea6d05 HTTP/1.1" 200 None 2025-09-28 05:15:57,405 - lava_callback - INFO - Completed processing callback for node 68d8bf95a178a5bbc0ea6d05 INFO: 172.21.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:16:14,260 - lava_callback - INFO - Received callback for node 68d6338dc1bc82a30b37ada0 INFO: 172.21.0.1:42404 - "POST /node/68d6338dc1bc82a30b37ada0 HTTP/1.0" 202 Accepted 2025-09-28 05:16:14,266 - lava_callback - INFO - Starting async job submit for node 68d6338dc1bc82a30b37ada0 2025-09-28 05:16:14,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:16:14,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6338dc1bc82a30b37ada0 HTTP/1.1" 200 None 2025-09-28 05:16:14,305 - lava_callback - DEBUG - Processing node 68d6338dc1bc82a30b37ada0: ltp-cap-bounds 2025-09-28 05:16:14,625 - lava_callback - ERROR - Error processing callback for node 68d6338dc1bc82a30b37ada0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38708 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:17:47,864 - lava_callback - INFO - Received callback for node 68d6338ec1bc82a30b37ada4 INFO: 172.21.0.1:38710 - "POST /node/68d6338ec1bc82a30b37ada4 HTTP/1.0" 202 Accepted 2025-09-28 05:17:47,872 - lava_callback - INFO - Starting async job submit for node 68d6338ec1bc82a30b37ada4 2025-09-28 05:17:47,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:17:47,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6338ec1bc82a30b37ada4 HTTP/1.1" 200 None 2025-09-28 05:17:47,984 - lava_callback - DEBUG - Processing node 68d6338ec1bc82a30b37ada4: ltp-containers 2025-09-28 05:17:48,793 - lava_callback - ERROR - Error processing callback for node 68d6338ec1bc82a30b37ada4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42468 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:18:42,220 - lava_callback - INFO - Received callback for node 68d8bf98a178a5bbc0ea6d0e INFO: 172.21.0.1:36062 - "POST /node/68d8bf98a178a5bbc0ea6d0e HTTP/1.0" 202 Accepted 2025-09-28 05:18:42,227 - lava_callback - INFO - Starting async job submit for node 68d8bf98a178a5bbc0ea6d0e 2025-09-28 05:18:42,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:18:42,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf98a178a5bbc0ea6d0e HTTP/1.1" 200 None 2025-09-28 05:18:42,291 - lava_callback - DEBUG - Processing node 68d8bf98a178a5bbc0ea6d0e: baseline-arm64 2025-09-28 05:18:42,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:18:42,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:18:42,931 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf98a178a5bbc0ea6d0e/log.txt.gz 2025-09-28 05:18:42,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:18:43,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:18:43,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bf98a178a5bbc0ea6d0e/lava_callback.json.gz DEBUG: Setup failed for 68d8bf98a178a5bbc0ea6d0e. Transitting job node result: incomplete -> fail 2025-09-28 05:18:43,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:18:43,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bf98a178a5bbc0ea6d0e HTTP/1.1" 200 None 2025-09-28 05:18:43,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:18:43,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:18:43,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:18:43,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bf98a178a5bbc0ea6d0e HTTP/1.1" 200 None 2025-09-28 05:18:43,215 - lava_callback - INFO - Completed processing callback for node 68d8bf98a178a5bbc0ea6d0e INFO: 172.21.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39972 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:19:03,820 - lava_callback - INFO - Received callback for node 68d8bfb4a178a5bbc0ea6d14 INFO: 172.21.0.1:42434 - "POST /node/68d8bfb4a178a5bbc0ea6d14 HTTP/1.0" 202 Accepted 2025-09-28 05:19:03,827 - lava_callback - INFO - Starting async job submit for node 68d8bfb4a178a5bbc0ea6d14 2025-09-28 05:19:03,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:19:03,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb4a178a5bbc0ea6d14 HTTP/1.1" 200 None 2025-09-28 05:19:03,900 - lava_callback - DEBUG - Processing node 68d8bfb4a178a5bbc0ea6d14: baseline-arm64 2025-09-28 05:19:05,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:19:05,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:19:05,185 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb4a178a5bbc0ea6d14/log.txt.gz 2025-09-28 05:19:05,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:19:05,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:19:05,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb4a178a5bbc0ea6d14/lava_callback.json.gz DEBUG: Setup failed for 68d8bfb4a178a5bbc0ea6d14. Transitting job node result: incomplete -> fail 2025-09-28 05:19:05,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:19:05,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb4a178a5bbc0ea6d14 HTTP/1.1" 200 None 2025-09-28 05:19:05,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:19:05,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:19:05,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:19:05,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bfb4a178a5bbc0ea6d14 HTTP/1.1" 200 None 2025-09-28 05:19:05,554 - lava_callback - INFO - Completed processing callback for node 68d8bfb4a178a5bbc0ea6d14 INFO: 172.21.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40562 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:19:54,614 - lava_callback - INFO - Received callback for node 68d61d5e30c58dac7d9423b1 INFO: 172.21.0.1:60784 - "POST /node/68d61d5e30c58dac7d9423b1 HTTP/1.0" 202 Accepted 2025-09-28 05:19:54,622 - lava_callback - INFO - Starting async job submit for node 68d61d5e30c58dac7d9423b1 2025-09-28 05:19:54,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:19:54,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d5e30c58dac7d9423b1 HTTP/1.1" 200 None 2025-09-28 05:19:54,739 - lava_callback - DEBUG - Processing node 68d61d5e30c58dac7d9423b1: ltp-cve 2025-09-28 05:19:55,498 - lava_callback - ERROR - Error processing callback for node 68d61d5e30c58dac7d9423b1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47818 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:22:05,181 - lava_callback - INFO - Received callback for node 68d7fe738dd6b39fd1aeb875 INFO: 172.21.0.1:38620 - "POST /node/68d7fe738dd6b39fd1aeb875 HTTP/1.0" 202 Accepted 2025-09-28 05:22:05,190 - lava_callback - INFO - Starting async job submit for node 68d7fe738dd6b39fd1aeb875 2025-09-28 05:22:05,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:22:05,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe738dd6b39fd1aeb875 HTTP/1.1" 200 None 2025-09-28 05:22:05,247 - lava_callback - DEBUG - Processing node 68d7fe738dd6b39fd1aeb875: baseline-arm64 2025-09-28 05:22:06,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:22:06,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:22:06,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe738dd6b39fd1aeb875/log.txt.gz 2025-09-28 05:22:06,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:22:06,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:22:06,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7fe738dd6b39fd1aeb875/lava_callback.json.gz DEBUG: Setup failed for 68d7fe738dd6b39fd1aeb875. Transitting job node result: incomplete -> fail 2025-09-28 05:22:06,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:22:06,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7fe738dd6b39fd1aeb875 HTTP/1.1" 200 None 2025-09-28 05:22:06,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:22:06,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:22:06,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:22:06,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7fe738dd6b39fd1aeb875 HTTP/1.1" 200 None 2025-09-28 05:22:06,675 - lava_callback - INFO - Completed processing callback for node 68d7fe738dd6b39fd1aeb875 INFO: 172.21.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52332 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:23:07,633 - lava_callback - INFO - Received callback for node 68d8b3c5a178a5bbc0ea69f0 INFO: 172.21.0.1:52334 - "POST /node/68d8b3c5a178a5bbc0ea69f0 HTTP/1.0" 202 Accepted 2025-09-28 05:23:07,639 - lava_callback - INFO - Starting async job submit for node 68d8b3c5a178a5bbc0ea69f0 2025-09-28 05:23:07,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:07,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b3c5a178a5bbc0ea69f0 HTTP/1.1" 200 None 2025-09-28 05:23:07,706 - lava_callback - DEBUG - Processing node 68d8b3c5a178a5bbc0ea69f0: baseline-arm 2025-09-28 05:23:07,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:23:08,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:23:08,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b3c5a178a5bbc0ea69f0/log.txt.gz 2025-09-28 05:23:08,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:23:08,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:23:08,257 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b3c5a178a5bbc0ea69f0/lava_callback.json.gz DEBUG: Setup failed for 68d8b3c5a178a5bbc0ea69f0. Transitting job node result: incomplete -> fail 2025-09-28 05:23:08,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:08,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b3c5a178a5bbc0ea69f0 HTTP/1.1" 200 None 2025-09-28 05:23:08,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:08,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 05:23:08,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:08,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b3c5a178a5bbc0ea69f0 HTTP/1.1" 200 None 2025-09-28 05:23:08,396 - lava_callback - INFO - Completed processing callback for node 68d8b3c5a178a5bbc0ea69f0 INFO: 172.21.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36748 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:23:23,838 - lava_callback - INFO - Received callback for node 68d8be03a178a5bbc0ea6cde INFO: 172.21.0.1:36754 - "POST /node/68d8be03a178a5bbc0ea6cde HTTP/1.0" 202 Accepted 2025-09-28 05:23:23,844 - lava_callback - INFO - Starting async job submit for node 68d8be03a178a5bbc0ea6cde 2025-09-28 05:23:23,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:23,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8be03a178a5bbc0ea6cde HTTP/1.1" 200 None 2025-09-28 05:23:23,885 - lava_callback - DEBUG - Processing node 68d8be03a178a5bbc0ea6cde: baseline-arm64-broonie 2025-09-28 05:23:23,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:23:24,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:23:24,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8be03a178a5bbc0ea6cde/log.txt.gz 2025-09-28 05:23:24,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:23:24,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:23:24,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8be03a178a5bbc0ea6cde/lava_callback.json.gz 2025-09-28 05:23:24,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:24,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8be03a178a5bbc0ea6cde HTTP/1.1" 200 None 2025-09-28 05:23:24,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:24,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:23:24,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:23:24,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8be03a178a5bbc0ea6cde HTTP/1.1" 200 None 2025-09-28 05:23:24,301 - lava_callback - INFO - Completed processing callback for node 68d8be03a178a5bbc0ea6cde INFO: 172.21.0.1:36762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53338 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:24:09,764 - lava_callback - INFO - Received callback for node 68d8c03da178a5bbc0ea6daf INFO: 172.21.0.1:41604 - "POST /node/68d8c03da178a5bbc0ea6daf HTTP/1.0" 202 Accepted 2025-09-28 05:24:09,772 - lava_callback - INFO - Starting async job submit for node 68d8c03da178a5bbc0ea6daf 2025-09-28 05:24:09,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:09,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c03da178a5bbc0ea6daf HTTP/1.1" 200 None 2025-09-28 05:24:09,841 - lava_callback - DEBUG - Processing node 68d8c03da178a5bbc0ea6daf: baseline-arm64 INFO: 172.21.0.1:41606 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:24:10,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:24:10,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:24:10,942 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c03da178a5bbc0ea6daf/log.txt.gz 2025-09-28 05:24:10,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:24:11,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:24:11,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c03da178a5bbc0ea6daf/lava_callback.json.gz DEBUG: Setup failed for 68d8c03da178a5bbc0ea6daf. Transitting job node result: incomplete -> fail 2025-09-28 05:24:11,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:11,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c03da178a5bbc0ea6daf HTTP/1.1" 200 None 2025-09-28 05:24:11,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:11,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:24:11,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:11,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c03da178a5bbc0ea6daf HTTP/1.1" 200 None 2025-09-28 05:24:11,290 - lava_callback - INFO - Completed processing callback for node 68d8c03da178a5bbc0ea6daf 2025-09-28 05:24:12,996 - lava_callback - INFO - Received callback for node 68d8bfb5a178a5bbc0ea6d17 INFO: 172.21.0.1:41608 - "POST /node/68d8bfb5a178a5bbc0ea6d17 HTTP/1.0" 202 Accepted 2025-09-28 05:24:13,003 - lava_callback - INFO - Starting async job submit for node 68d8bfb5a178a5bbc0ea6d17 2025-09-28 05:24:13,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:13,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb5a178a5bbc0ea6d17 HTTP/1.1" 200 None 2025-09-28 05:24:13,057 - lava_callback - DEBUG - Processing node 68d8bfb5a178a5bbc0ea6d17: baseline-arm64 2025-09-28 05:24:14,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:24:14,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:24:14,247 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb5a178a5bbc0ea6d17/log.txt.gz 2025-09-28 05:24:14,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:24:14,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:24:14,423 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb5a178a5bbc0ea6d17/lava_callback.json.gz DEBUG: Setup failed for 68d8bfb5a178a5bbc0ea6d17. Transitting job node result: incomplete -> fail 2025-09-28 05:24:14,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:14,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb5a178a5bbc0ea6d17 HTTP/1.1" 200 None 2025-09-28 05:24:14,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:14,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:24:14,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:24:14,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bfb5a178a5bbc0ea6d17 HTTP/1.1" 200 None 2025-09-28 05:24:14,554 - lava_callback - INFO - Completed processing callback for node 68d8bfb5a178a5bbc0ea6d17 INFO: 172.21.0.1:41612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55972 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:27:20,295 - lava_callback - INFO - Received callback for node 68d8c081a178a5bbc0ea6db8 INFO: 172.21.0.1:49780 - "POST /node/68d8c081a178a5bbc0ea6db8 HTTP/1.0" 202 Accepted 2025-09-28 05:27:20,303 - lava_callback - INFO - Starting async job submit for node 68d8c081a178a5bbc0ea6db8 2025-09-28 05:27:20,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:20,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c081a178a5bbc0ea6db8 HTTP/1.1" 200 None 2025-09-28 05:27:20,376 - lava_callback - DEBUG - Processing node 68d8c081a178a5bbc0ea6db8: baseline-arm64 2025-09-28 05:27:21,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:27:21,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:27:21,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c081a178a5bbc0ea6db8/log.txt.gz 2025-09-28 05:27:21,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:27:21,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:27:21,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c081a178a5bbc0ea6db8/lava_callback.json.gz DEBUG: Setup failed for 68d8c081a178a5bbc0ea6db8. Transitting job node result: incomplete -> fail 2025-09-28 05:27:21,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:21,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c081a178a5bbc0ea6db8 HTTP/1.1" 200 None 2025-09-28 05:27:21,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:21,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:27:21,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:49790 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:27:21,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c081a178a5bbc0ea6db8 HTTP/1.1" 200 None 2025-09-28 05:27:21,771 - lava_callback - INFO - Completed processing callback for node 68d8c081a178a5bbc0ea6db8 INFO: 172.21.0.1:49802 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:27:36,253 - lava_callback - INFO - Received callback for node 68d8b6fea178a5bbc0ea6b78 INFO: 172.21.0.1:34844 - "POST /node/68d8b6fea178a5bbc0ea6b78 HTTP/1.0" 202 Accepted 2025-09-28 05:27:36,299 - lava_callback - INFO - Starting async job submit for node 68d8b6fea178a5bbc0ea6b78 2025-09-28 05:27:36,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:36,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b6fea178a5bbc0ea6b78 HTTP/1.1" 200 None 2025-09-28 05:27:36,356 - lava_callback - DEBUG - Processing node 68d8b6fea178a5bbc0ea6b78: baseline-x86 2025-09-28 05:27:36,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:27:36,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:27:36,605 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b6fea178a5bbc0ea6b78/log.txt.gz 2025-09-28 05:27:36,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:27:36,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:27:36,760 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b6fea178a5bbc0ea6b78/lava_callback.json.gz 2025-09-28 05:27:36,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:36,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b6fea178a5bbc0ea6b78 HTTP/1.1" 200 None 2025-09-28 05:27:36,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:36,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f5b7aff7facd9a8d14f HTTP/1.1" 200 None 2025-09-28 05:27:36,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:36,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b6fea178a5bbc0ea6b78 HTTP/1.1" 200 None 2025-09-28 05:27:36,851 - lava_callback - INFO - Completed processing callback for node 68d8b6fea178a5bbc0ea6b78 INFO: 172.21.0.1:32822 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:27:53,616 - lava_callback - INFO - Received callback for node 68d7feeb8dd6b39fd1aeb883 INFO: 172.21.0.1:48042 - "POST /node/68d7feeb8dd6b39fd1aeb883 HTTP/1.0" 202 Accepted 2025-09-28 05:27:53,623 - lava_callback - INFO - Starting async job submit for node 68d7feeb8dd6b39fd1aeb883 2025-09-28 05:27:53,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:53,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7feeb8dd6b39fd1aeb883 HTTP/1.1" 200 None 2025-09-28 05:27:53,698 - lava_callback - DEBUG - Processing node 68d7feeb8dd6b39fd1aeb883: baseline-arm64 2025-09-28 05:27:54,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:27:54,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:27:54,762 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7feeb8dd6b39fd1aeb883/log.txt.gz 2025-09-28 05:27:54,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:27:54,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:27:54,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7feeb8dd6b39fd1aeb883/lava_callback.json.gz DEBUG: Setup failed for 68d7feeb8dd6b39fd1aeb883. Transitting job node result: incomplete -> fail 2025-09-28 05:27:55,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:55,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7feeb8dd6b39fd1aeb883 HTTP/1.1" 200 None 2025-09-28 05:27:55,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:55,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:27:55,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:27:55,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7feeb8dd6b39fd1aeb883 HTTP/1.1" 200 None 2025-09-28 05:27:55,099 - lava_callback - INFO - Completed processing callback for node 68d7feeb8dd6b39fd1aeb883 INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33794 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:28:13,672 - lava_callback - INFO - Received callback for node 68d7feec8dd6b39fd1aeb886 INFO: 172.21.0.1:33800 - "POST /node/68d7feec8dd6b39fd1aeb886 HTTP/1.0" 202 Accepted 2025-09-28 05:28:13,679 - lava_callback - INFO - Starting async job submit for node 68d7feec8dd6b39fd1aeb886 2025-09-28 05:28:13,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:28:13,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7feec8dd6b39fd1aeb886 HTTP/1.1" 200 None 2025-09-28 05:28:13,746 - lava_callback - DEBUG - Processing node 68d7feec8dd6b39fd1aeb886: baseline-arm64 2025-09-28 05:28:14,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:28:14,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:28:14,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7feec8dd6b39fd1aeb886/log.txt.gz 2025-09-28 05:28:14,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:28:14,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:28:14,958 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d7feec8dd6b39fd1aeb886/lava_callback.json.gz DEBUG: Setup failed for 68d7feec8dd6b39fd1aeb886. Transitting job node result: incomplete -> fail 2025-09-28 05:28:15,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:28:15,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d7feec8dd6b39fd1aeb886 HTTP/1.1" 200 None 2025-09-28 05:28:15,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:28:15,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:28:15,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:28:15,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d7feec8dd6b39fd1aeb886 HTTP/1.1" 200 None 2025-09-28 05:28:15,110 - lava_callback - INFO - Completed processing callback for node 68d7feec8dd6b39fd1aeb886 INFO: 172.21.0.1:33802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56998 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:31:53,719 - lava_callback - INFO - Received callback for node 68d8c03da178a5bbc0ea6db2 INFO: 172.21.0.1:55610 - "POST /node/68d8c03da178a5bbc0ea6db2 HTTP/1.0" 202 Accepted 2025-09-28 05:31:53,726 - lava_callback - INFO - Starting async job submit for node 68d8c03da178a5bbc0ea6db2 2025-09-28 05:31:53,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:31:53,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c03da178a5bbc0ea6db2 HTTP/1.1" 200 None 2025-09-28 05:31:53,779 - lava_callback - DEBUG - Processing node 68d8c03da178a5bbc0ea6db2: baseline-arm64 2025-09-28 05:31:54,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:31:54,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:31:54,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c03da178a5bbc0ea6db2/log.txt.gz 2025-09-28 05:31:54,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:31:55,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:31:55,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c03da178a5bbc0ea6db2/lava_callback.json.gz DEBUG: Setup failed for 68d8c03da178a5bbc0ea6db2. Transitting job node result: incomplete -> fail 2025-09-28 05:31:55,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:31:55,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c03da178a5bbc0ea6db2 HTTP/1.1" 200 None 2025-09-28 05:31:55,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:31:55,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:31:55,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:31:55,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c03da178a5bbc0ea6db2 HTTP/1.1" 200 None 2025-09-28 05:31:55,299 - lava_callback - INFO - Completed processing callback for node 68d8c03da178a5bbc0ea6db2 INFO: 172.21.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48698 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:32:35,405 - lava_callback - INFO - Received callback for node 68d6338ec1bc82a30b37ada7 INFO: 172.21.0.1:55114 - "POST /node/68d6338ec1bc82a30b37ada7 HTTP/1.0" 202 Accepted 2025-09-28 05:32:35,416 - lava_callback - INFO - Starting async job submit for node 68d6338ec1bc82a30b37ada7 2025-09-28 05:32:35,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:32:35,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6338ec1bc82a30b37ada7 HTTP/1.1" 200 None 2025-09-28 05:32:35,769 - lava_callback - DEBUG - Processing node 68d6338ec1bc82a30b37ada7: ltp-controllers INFO: 172.21.0.1:55130 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:32:38,728 - lava_callback - ERROR - Error processing callback for node 68d6338ec1bc82a30b37ada7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58652 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:33:21,256 - lava_callback - INFO - Received callback for node 68d8c081a178a5bbc0ea6dbb INFO: 172.21.0.1:43608 - "POST /node/68d8c081a178a5bbc0ea6dbb HTTP/1.0" 202 Accepted 2025-09-28 05:33:21,271 - lava_callback - INFO - Starting async job submit for node 68d8c081a178a5bbc0ea6dbb 2025-09-28 05:33:21,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:33:21,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c081a178a5bbc0ea6dbb HTTP/1.1" 200 None 2025-09-28 05:33:21,345 - lava_callback - DEBUG - Processing node 68d8c081a178a5bbc0ea6dbb: baseline-arm64 2025-09-28 05:33:22,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:33:22,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:33:22,472 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c081a178a5bbc0ea6dbb/log.txt.gz 2025-09-28 05:33:22,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:33:22,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:33:22,665 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c081a178a5bbc0ea6dbb/lava_callback.json.gz DEBUG: Setup failed for 68d8c081a178a5bbc0ea6dbb. Transitting job node result: incomplete -> fail 2025-09-28 05:33:22,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:33:22,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c081a178a5bbc0ea6dbb HTTP/1.1" 200 None 2025-09-28 05:33:22,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:33:22,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:33:22,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:33:22,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c081a178a5bbc0ea6dbb HTTP/1.1" 200 None 2025-09-28 05:33:22,796 - lava_callback - INFO - Completed processing callback for node 68d8c081a178a5bbc0ea6dbb INFO: 172.21.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47196 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:35:24,890 - lava_callback - INFO - Received callback for node 68d8bfb7a178a5bbc0ea6d1d INFO: 172.21.0.1:47200 - "POST /node/68d8bfb7a178a5bbc0ea6d1d HTTP/1.0" 202 Accepted 2025-09-28 05:35:24,897 - lava_callback - INFO - Starting async job submit for node 68d8bfb7a178a5bbc0ea6d1d 2025-09-28 05:35:24,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:35:24,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb7a178a5bbc0ea6d1d HTTP/1.1" 200 None 2025-09-28 05:35:24,985 - lava_callback - DEBUG - Processing node 68d8bfb7a178a5bbc0ea6d1d: baseline-arm64 2025-09-28 05:35:26,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:35:26,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:35:26,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb7a178a5bbc0ea6d1d/log.txt.gz 2025-09-28 05:35:26,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:47206 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:35:26,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:35:26,550 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb7a178a5bbc0ea6d1d/lava_callback.json.gz DEBUG: Setup failed for 68d8bfb7a178a5bbc0ea6d1d. Transitting job node result: incomplete -> fail 2025-09-28 05:35:26,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:35:26,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb7a178a5bbc0ea6d1d HTTP/1.1" 200 None 2025-09-28 05:35:26,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:35:26,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:35:26,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:35:26,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bfb7a178a5bbc0ea6d1d HTTP/1.1" 200 None 2025-09-28 05:35:26,689 - lava_callback - INFO - Completed processing callback for node 68d8bfb7a178a5bbc0ea6d1d INFO: 172.21.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47706 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:36:34,569 - lava_callback - INFO - Received callback for node 68d8bfb8a178a5bbc0ea6d23 INFO: 172.21.0.1:46002 - "POST /node/68d8bfb8a178a5bbc0ea6d23 HTTP/1.0" 202 Accepted 2025-09-28 05:36:34,576 - lava_callback - INFO - Starting async job submit for node 68d8bfb8a178a5bbc0ea6d23 2025-09-28 05:36:34,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:36:34,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb8a178a5bbc0ea6d23 HTTP/1.1" 200 None 2025-09-28 05:36:34,640 - lava_callback - DEBUG - Processing node 68d8bfb8a178a5bbc0ea6d23: baseline-arm64 2025-09-28 05:36:35,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:36:35,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:36:35,855 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb8a178a5bbc0ea6d23/log.txt.gz 2025-09-28 05:36:35,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:36:35,957 - lava_callback - INFO - Received callback for node 68d8bfbba178a5bbc0ea6d26 INFO: 172.21.0.1:46018 - "POST /node/68d8bfbba178a5bbc0ea6d26 HTTP/1.0" 202 Accepted 2025-09-28 05:36:35,965 - lava_callback - INFO - Starting async job submit for node 68d8bfbba178a5bbc0ea6d26 2025-09-28 05:36:36,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:36:36,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfbba178a5bbc0ea6d26 HTTP/1.1" 200 None 2025-09-28 05:36:36,019 - lava_callback - DEBUG - Processing node 68d8bfbba178a5bbc0ea6d26: baseline-arm64 2025-09-28 05:36:36,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:36:36,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfb8a178a5bbc0ea6d23/lava_callback.json.gz DEBUG: Setup failed for 68d8bfb8a178a5bbc0ea6d23. Transitting job node result: incomplete -> fail 2025-09-28 05:36:36,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:36:36,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfb8a178a5bbc0ea6d23 HTTP/1.1" 200 None 2025-09-28 05:36:36,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:36:36,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:36:36,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:46020 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:36:36,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bfb8a178a5bbc0ea6d23 HTTP/1.1" 200 None 2025-09-28 05:36:36,579 - lava_callback - INFO - Completed processing callback for node 68d8bfb8a178a5bbc0ea6d23 2025-09-28 05:36:37,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:36:37,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:36:37,582 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfbba178a5bbc0ea6d26/log.txt.gz 2025-09-28 05:36:37,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:36:37,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:36:37,760 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfbba178a5bbc0ea6d26/lava_callback.json.gz DEBUG: Setup failed for 68d8bfbba178a5bbc0ea6d26. Transitting job node result: incomplete -> fail 2025-09-28 05:36:37,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:36:37,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfbba178a5bbc0ea6d26 HTTP/1.1" 200 None 2025-09-28 05:36:37,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:36:37,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:36:37,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:36:37,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bfbba178a5bbc0ea6d26 HTTP/1.1" 200 None 2025-09-28 05:36:37,925 - lava_callback - INFO - Completed processing callback for node 68d8bfbba178a5bbc0ea6d26 INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35664 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:37:15,065 - lava_callback - INFO - Received callback for node 68d63391c1bc82a30b37adb6 INFO: 172.21.0.1:35676 - "POST /node/68d63391c1bc82a30b37adb6 HTTP/1.0" 202 Accepted 2025-09-28 05:37:15,071 - lava_callback - INFO - Starting async job submit for node 68d63391c1bc82a30b37adb6 2025-09-28 05:37:15,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:37:15,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63391c1bc82a30b37adb6 HTTP/1.1" 200 None 2025-09-28 05:37:15,110 - lava_callback - DEBUG - Processing node 68d63391c1bc82a30b37adb6: ltp-fsx INFO: 172.21.0.1:35692 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 05:37:15,471 - lava_callback - ERROR - Error processing callback for node 68d63391c1bc82a30b37adb6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:38:30,460 - lava_callback - INFO - Received callback for node 68d8c1bba178a5bbc0ea6dd1 INFO: 172.21.0.1:35604 - "POST /node/68d8c1bba178a5bbc0ea6dd1 HTTP/1.0" 202 Accepted 2025-09-28 05:38:30,469 - lava_callback - INFO - Starting async job submit for node 68d8c1bba178a5bbc0ea6dd1 2025-09-28 05:38:30,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:38:30,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1bba178a5bbc0ea6dd1 HTTP/1.1" 200 None 2025-09-28 05:38:30,519 - lava_callback - DEBUG - Processing node 68d8c1bba178a5bbc0ea6dd1: baseline-arm64 2025-09-28 05:38:31,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:38:31,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:38:31,644 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c1bba178a5bbc0ea6dd1/log.txt.gz 2025-09-28 05:38:31,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:38:31,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:38:31,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c1bba178a5bbc0ea6dd1/lava_callback.json.gz DEBUG: Setup failed for 68d8c1bba178a5bbc0ea6dd1. Transitting job node result: incomplete -> fail 2025-09-28 05:38:31,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:38:31,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1bba178a5bbc0ea6dd1 HTTP/1.1" 200 None 2025-09-28 05:38:31,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:38:31,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:38:31,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:38:31,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c1bba178a5bbc0ea6dd1 HTTP/1.1" 200 None 2025-09-28 05:38:31,961 - lava_callback - INFO - Completed processing callback for node 68d8c1bba178a5bbc0ea6dd1 INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47798 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:38:55,350 - lava_callback - INFO - Received callback for node 68d63390c1bc82a30b37adb3 INFO: 172.21.0.1:39464 - "POST /node/68d63390c1bc82a30b37adb3 HTTP/1.0" 202 Accepted 2025-09-28 05:38:55,358 - lava_callback - INFO - Starting async job submit for node 68d63390c1bc82a30b37adb3 2025-09-28 05:38:55,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:38:55,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63390c1bc82a30b37adb3 HTTP/1.1" 200 None 2025-09-28 05:38:55,471 - lava_callback - DEBUG - Processing node 68d63390c1bc82a30b37adb3: ltp-fs-bind 2025-09-28 05:38:56,021 - lava_callback - ERROR - Error processing callback for node 68d63390c1bc82a30b37adb3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56702 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:41:35,224 - lava_callback - INFO - Received callback for node 68d63391c1bc82a30b37adb9 INFO: 172.21.0.1:38908 - "POST /node/68d63391c1bc82a30b37adb9 HTTP/1.0" 202 Accepted 2025-09-28 05:41:35,231 - lava_callback - INFO - Starting async job submit for node 68d63391c1bc82a30b37adb9 2025-09-28 05:41:35,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:41:35,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63391c1bc82a30b37adb9 HTTP/1.1" 200 None 2025-09-28 05:41:35,283 - lava_callback - DEBUG - Processing node 68d63391c1bc82a30b37adb9: ltp-input 2025-09-28 05:41:35,649 - lava_callback - ERROR - Error processing callback for node 68d63391c1bc82a30b37adb9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34878 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:42:25,186 - lava_callback - INFO - Received callback for node 68d8bfbba178a5bbc0ea6d29 INFO: 172.21.0.1:47118 - "POST /node/68d8bfbba178a5bbc0ea6d29 HTTP/1.0" 202 Accepted 2025-09-28 05:42:25,192 - lava_callback - INFO - Starting async job submit for node 68d8bfbba178a5bbc0ea6d29 2025-09-28 05:42:25,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:25,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfbba178a5bbc0ea6d29 HTTP/1.1" 200 None 2025-09-28 05:42:25,238 - lava_callback - DEBUG - Processing node 68d8bfbba178a5bbc0ea6d29: baseline-arm64 INFO: 172.21.0.1:47130 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:42:25,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:42:25,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:42:25,796 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfbba178a5bbc0ea6d29/log.txt.gz 2025-09-28 05:42:25,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:42:25,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:42:25,943 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8bfbba178a5bbc0ea6d29/lava_callback.json.gz DEBUG: Setup failed for 68d8bfbba178a5bbc0ea6d29. Transitting job node result: incomplete -> fail 2025-09-28 05:42:26,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:26,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bfbba178a5bbc0ea6d29 HTTP/1.1" 200 None 2025-09-28 05:42:26,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:26,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:42:26,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:26,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bfbba178a5bbc0ea6d29 HTTP/1.1" 200 None 2025-09-28 05:42:26,069 - lava_callback - INFO - Completed processing callback for node 68d8bfbba178a5bbc0ea6d29 INFO: 172.21.0.1:47146 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:42:26,981 - lava_callback - INFO - Received callback for node 68d803418dd6b39fd1aeb8e2 INFO: 172.21.0.1:47160 - "POST /node/68d803418dd6b39fd1aeb8e2 HTTP/1.0" 202 Accepted 2025-09-28 05:42:26,989 - lava_callback - INFO - Starting async job submit for node 68d803418dd6b39fd1aeb8e2 2025-09-28 05:42:27,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:27,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d803418dd6b39fd1aeb8e2 HTTP/1.1" 200 None 2025-09-28 05:42:27,062 - lava_callback - DEBUG - Processing node 68d803418dd6b39fd1aeb8e2: baseline-arm64 2025-09-28 05:42:28,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:42:28,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:42:28,308 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d803418dd6b39fd1aeb8e2/log.txt.gz 2025-09-28 05:42:28,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:42:28,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:42:28,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d803418dd6b39fd1aeb8e2/lava_callback.json.gz DEBUG: Setup failed for 68d803418dd6b39fd1aeb8e2. Transitting job node result: incomplete -> fail 2025-09-28 05:42:28,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:28,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d803418dd6b39fd1aeb8e2 HTTP/1.1" 200 None 2025-09-28 05:42:28,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:28,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:42:28,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:28,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d803418dd6b39fd1aeb8e2 HTTP/1.1" 200 None 2025-09-28 05:42:28,661 - lava_callback - INFO - Completed processing callback for node 68d803418dd6b39fd1aeb8e2 INFO: 172.21.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38542 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:42:48,845 - lava_callback - INFO - Received callback for node 68d8b3c6a178a5bbc0ea69f3 INFO: 172.21.0.1:38546 - "POST /node/68d8b3c6a178a5bbc0ea69f3 HTTP/1.0" 202 Accepted 2025-09-28 05:42:48,850 - lava_callback - INFO - Starting async job submit for node 68d8b3c6a178a5bbc0ea69f3 2025-09-28 05:42:48,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:48,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b3c6a178a5bbc0ea69f3 HTTP/1.1" 200 None 2025-09-28 05:42:48,893 - lava_callback - DEBUG - Processing node 68d8b3c6a178a5bbc0ea69f3: baseline-arm 2025-09-28 05:42:49,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:42:49,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:42:49,281 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b3c6a178a5bbc0ea69f3/log.txt.gz 2025-09-28 05:42:49,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:42:49,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:42:49,418 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b3c6a178a5bbc0ea69f3/lava_callback.json.gz DEBUG: Setup failed for 68d8b3c6a178a5bbc0ea69f3. Transitting job node result: incomplete -> fail 2025-09-28 05:42:49,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:49,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b3c6a178a5bbc0ea69f3 HTTP/1.1" 200 None 2025-09-28 05:42:49,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:49,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 05:42:49,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:42:49,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b3c6a178a5bbc0ea69f3 HTTP/1.1" 200 None 2025-09-28 05:42:49,543 - lava_callback - INFO - Completed processing callback for node 68d8b3c6a178a5bbc0ea69f3 INFO: 172.21.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53628 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 05:43:15,328 - lava_callback - INFO - Received callback for node 68d63392c1bc82a30b37adbe INFO: 172.21.0.1:53632 - "POST /node/68d63392c1bc82a30b37adbe HTTP/1.0" 202 Accepted 2025-09-28 05:43:15,334 - lava_callback - INFO - Starting async job submit for node 68d63392c1bc82a30b37adbe 2025-09-28 05:43:15,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:43:15,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63392c1bc82a30b37adbe HTTP/1.1" 200 None 2025-09-28 05:43:15,384 - lava_callback - DEBUG - Processing node 68d63392c1bc82a30b37adbe: ltp-ipc 2025-09-28 05:43:15,734 - lava_callback - ERROR - Error processing callback for node 68d63392c1bc82a30b37adbe: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:44:06,254 - lava_callback - INFO - Received callback for node 68d6338fc1bc82a30b37adaa INFO: 172.21.0.1:45892 - "POST /node/68d6338fc1bc82a30b37adaa HTTP/1.0" 202 Accepted 2025-09-28 05:44:06,261 - lava_callback - INFO - Starting async job submit for node 68d6338fc1bc82a30b37adaa 2025-09-28 05:44:06,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:44:06,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6338fc1bc82a30b37adaa HTTP/1.1" 200 None 2025-09-28 05:44:06,399 - lava_callback - DEBUG - Processing node 68d6338fc1bc82a30b37adaa: ltp-cve INFO: 172.21.0.1:45896 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:44:07,172 - lava_callback - ERROR - Error processing callback for node 68d6338fc1bc82a30b37adaa: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:44:54,206 - lava_callback - INFO - Received callback for node 68d8c1bca178a5bbc0ea6dd4 INFO: 172.21.0.1:59738 - "POST /node/68d8c1bca178a5bbc0ea6dd4 HTTP/1.0" 202 Accepted 2025-09-28 05:44:54,214 - lava_callback - INFO - Starting async job submit for node 68d8c1bca178a5bbc0ea6dd4 2025-09-28 05:44:54,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:44:54,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1bca178a5bbc0ea6dd4 HTTP/1.1" 200 None 2025-09-28 05:44:54,266 - lava_callback - DEBUG - Processing node 68d8c1bca178a5bbc0ea6dd4: baseline-arm64 2025-09-28 05:44:55,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:44:55,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:44:55,408 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c1bca178a5bbc0ea6dd4/log.txt.gz 2025-09-28 05:44:55,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:44:55,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:44:55,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c1bca178a5bbc0ea6dd4/lava_callback.json.gz DEBUG: Setup failed for 68d8c1bca178a5bbc0ea6dd4. Transitting job node result: incomplete -> fail 2025-09-28 05:44:55,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:44:55,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1bca178a5bbc0ea6dd4 HTTP/1.1" 200 None 2025-09-28 05:44:55,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:44:55,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:44:55,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:44:55,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c1bca178a5bbc0ea6dd4 HTTP/1.1" 200 None 2025-09-28 05:44:55,825 - lava_callback - INFO - Completed processing callback for node 68d8c1bca178a5bbc0ea6dd4 INFO: 172.21.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33504 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:47:45,559 - lava_callback - INFO - Received callback for node 68d8c320a178a5bbc0ea6deb INFO: 172.21.0.1:43920 - "POST /node/68d8c320a178a5bbc0ea6deb HTTP/1.0" 202 Accepted 2025-09-28 05:47:45,567 - lava_callback - INFO - Starting async job submit for node 68d8c320a178a5bbc0ea6deb 2025-09-28 05:47:45,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:47:45,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c320a178a5bbc0ea6deb HTTP/1.1" 200 None 2025-09-28 05:47:45,620 - lava_callback - DEBUG - Processing node 68d8c320a178a5bbc0ea6deb: baseline-arm64 INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:47:46,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:47:46,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:47:46,910 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c320a178a5bbc0ea6deb/log.txt.gz 2025-09-28 05:47:46,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:47:47,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:47:47,115 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c320a178a5bbc0ea6deb/lava_callback.json.gz DEBUG: Setup failed for 68d8c320a178a5bbc0ea6deb. Transitting job node result: incomplete -> fail 2025-09-28 05:47:47,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:47:47,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c320a178a5bbc0ea6deb HTTP/1.1" 200 None 2025-09-28 05:47:47,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:47:47,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:47:47,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:47:47,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c320a178a5bbc0ea6deb HTTP/1.1" 200 None 2025-09-28 05:47:47,247 - lava_callback - INFO - Completed processing callback for node 68d8c320a178a5bbc0ea6deb INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:47:56,577 - lava_callback - INFO - Received callback for node 68d61d6d30c58dac7d9423cc INFO: 172.21.0.1:46586 - "POST /node/68d61d6d30c58dac7d9423cc HTTP/1.0" 202 Accepted 2025-09-28 05:47:56,596 - lava_callback - INFO - Starting async job submit for node 68d61d6d30c58dac7d9423cc 2025-09-28 05:47:57,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:47:57,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d61d6d30c58dac7d9423cc HTTP/1.1" 200 None 2025-09-28 05:47:57,954 - lava_callback - DEBUG - Processing node 68d61d6d30c58dac7d9423cc: ltp-syscalls 2025-09-28 05:48:04,806 - lava_callback - ERROR - Error processing callback for node 68d61d6d30c58dac7d9423cc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39050 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:48:31,611 - lava_callback - INFO - Received callback for node 68d8c1c2a178a5bbc0ea6dd7 INFO: 172.21.0.1:59316 - "POST /node/68d8c1c2a178a5bbc0ea6dd7 HTTP/1.0" 202 Accepted 2025-09-28 05:48:31,659 - lava_callback - INFO - Starting async job submit for node 68d8c1c2a178a5bbc0ea6dd7 2025-09-28 05:48:31,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:31,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1c2a178a5bbc0ea6dd7 HTTP/1.1" 200 None 2025-09-28 05:48:31,701 - lava_callback - DEBUG - Processing node 68d8c1c2a178a5bbc0ea6dd7: baseline-x86 2025-09-28 05:48:31,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:48:31,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:48:31,950 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8c1c2a178a5bbc0ea6dd7/log.txt.gz 2025-09-28 05:48:31,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:48:32,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:48:32,081 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8c1c2a178a5bbc0ea6dd7/lava_callback.json.gz 2025-09-28 05:48:32,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:32,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1c2a178a5bbc0ea6dd7 HTTP/1.1" 200 None 2025-09-28 05:48:32,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:32,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aafca178a5bbc0ea5cd2 HTTP/1.1" 200 None 2025-09-28 05:48:32,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:32,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c1c2a178a5bbc0ea6dd7 HTTP/1.1" 200 None 2025-09-28 05:48:32,171 - lava_callback - INFO - Completed processing callback for node 68d8c1c2a178a5bbc0ea6dd7 2025-09-28 05:48:33,881 - lava_callback - INFO - Received callback for node 68d803428dd6b39fd1aeb8e5 INFO: 172.21.0.1:59324 - "POST /node/68d803428dd6b39fd1aeb8e5 HTTP/1.0" 202 Accepted 2025-09-28 05:48:33,890 - lava_callback - INFO - Starting async job submit for node 68d803428dd6b39fd1aeb8e5 2025-09-28 05:48:33,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:33,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d803428dd6b39fd1aeb8e5 HTTP/1.1" 200 None 2025-09-28 05:48:33,953 - lava_callback - DEBUG - Processing node 68d803428dd6b39fd1aeb8e5: baseline-arm64 2025-09-28 05:48:35,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:48:35,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:48:35,280 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d803428dd6b39fd1aeb8e5/log.txt.gz 2025-09-28 05:48:35,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:48:35,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:48:35,488 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d803428dd6b39fd1aeb8e5/lava_callback.json.gz DEBUG: Setup failed for 68d803428dd6b39fd1aeb8e5. Transitting job node result: incomplete -> fail 2025-09-28 05:48:35,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:35,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d803428dd6b39fd1aeb8e5 HTTP/1.1" 200 None 2025-09-28 05:48:35,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:35,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:48:35,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:35,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d803428dd6b39fd1aeb8e5 HTTP/1.1" 200 None 2025-09-28 05:48:35,651 - lava_callback - INFO - Completed processing callback for node 68d803428dd6b39fd1aeb8e5 INFO: 172.21.0.1:59330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49464 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:48:54,400 - lava_callback - INFO - Received callback for node 68d8034c8dd6b39fd1aeb8eb INFO: 172.21.0.1:37554 - "POST /node/68d8034c8dd6b39fd1aeb8eb HTTP/1.0" 202 Accepted 2025-09-28 05:48:54,407 - lava_callback - INFO - Starting async job submit for node 68d8034c8dd6b39fd1aeb8eb 2025-09-28 05:48:54,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:54,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8034c8dd6b39fd1aeb8eb HTTP/1.1" 200 None 2025-09-28 05:48:54,479 - lava_callback - DEBUG - Processing node 68d8034c8dd6b39fd1aeb8eb: baseline-arm64 2025-09-28 05:48:55,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:48:55,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:48:55,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8034c8dd6b39fd1aeb8eb/log.txt.gz 2025-09-28 05:48:55,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:48:55,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:48:55,707 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8034c8dd6b39fd1aeb8eb/lava_callback.json.gz DEBUG: Setup failed for 68d8034c8dd6b39fd1aeb8eb. Transitting job node result: incomplete -> fail 2025-09-28 05:48:55,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:55,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8034c8dd6b39fd1aeb8eb HTTP/1.1" 200 None 2025-09-28 05:48:55,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:55,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 05:48:55,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:48:55,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8034c8dd6b39fd1aeb8eb HTTP/1.1" 200 None 2025-09-28 05:48:55,846 - lava_callback - INFO - Completed processing callback for node 68d8034c8dd6b39fd1aeb8eb INFO: 172.21.0.1:37556 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:49:01,029 - lava_callback - INFO - Received callback for node 68d8c371a178a5bbc0ea6df4 INFO: 172.21.0.1:47858 - "POST /node/68d8c371a178a5bbc0ea6df4 HTTP/1.0" 202 Accepted 2025-09-28 05:49:01,037 - lava_callback - INFO - Starting async job submit for node 68d8c371a178a5bbc0ea6df4 2025-09-28 05:49:01,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:49:01,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c371a178a5bbc0ea6df4 HTTP/1.1" 200 None 2025-09-28 05:49:01,086 - lava_callback - DEBUG - Processing node 68d8c371a178a5bbc0ea6df4: baseline-arm64 2025-09-28 05:49:02,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:49:02,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:49:02,214 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c371a178a5bbc0ea6df4/log.txt.gz 2025-09-28 05:49:02,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:49:02,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:49:02,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c371a178a5bbc0ea6df4/lava_callback.json.gz DEBUG: Setup failed for 68d8c371a178a5bbc0ea6df4. Transitting job node result: incomplete -> fail 2025-09-28 05:49:02,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:49:02,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c371a178a5bbc0ea6df4 HTTP/1.1" 200 None 2025-09-28 05:49:02,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:49:02,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:49:02,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:49:02,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c371a178a5bbc0ea6df4 HTTP/1.1" 200 None 2025-09-28 05:49:02,537 - lava_callback - INFO - Completed processing callback for node 68d8c371a178a5bbc0ea6df4 INFO: 172.21.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:49:55,381 - lava_callback - INFO - Received callback for node 68d63381c1bc82a30b37ad3c INFO: 172.21.0.1:34562 - "POST /node/68d63381c1bc82a30b37ad3c HTTP/1.0" 202 Accepted 2025-09-28 05:49:55,387 - lava_callback - INFO - Starting async job submit for node 68d63381c1bc82a30b37ad3c 2025-09-28 05:49:55,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:49:55,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63381c1bc82a30b37ad3c HTTP/1.1" 200 None 2025-09-28 05:49:55,441 - lava_callback - DEBUG - Processing node 68d63381c1bc82a30b37ad3c: baseline-arm-broonie 2025-09-28 05:49:55,653 - lava_callback - ERROR - Error processing callback for node 68d63381c1bc82a30b37ad3c: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34564 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:50:03,758 - lava_callback - INFO - Received callback for node 68d811e01fd44e101337f107 INFO: 172.21.0.1:51546 - "POST /node/68d811e01fd44e101337f107 HTTP/1.0" 202 Accepted 2025-09-28 05:50:03,764 - lava_callback - INFO - Starting async job submit for node 68d811e01fd44e101337f107 2025-09-28 05:50:03,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:03,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811e01fd44e101337f107 HTTP/1.1" 200 None 2025-09-28 05:50:03,818 - lava_callback - DEBUG - Processing node 68d811e01fd44e101337f107: baseline-arm64 2025-09-28 05:50:04,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:50:04,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:50:04,306 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811e01fd44e101337f107/log.txt.gz 2025-09-28 05:50:04,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:50:04,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:50:04,442 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811e01fd44e101337f107/lava_callback.json.gz 2025-09-28 05:50:04,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:04,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811e01fd44e101337f107 HTTP/1.1" 200 None 2025-09-28 05:50:04,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:04,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d80b5e8dd6b39fd1aeb97b HTTP/1.1" 200 None 2025-09-28 05:50:04,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:04,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d811e01fd44e101337f107 HTTP/1.1" 200 None 2025-09-28 05:50:04,539 - lava_callback - INFO - Completed processing callback for node 68d811e01fd44e101337f107 INFO: 172.21.0.1:51548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46640 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:50:23,562 - lava_callback - INFO - Received callback for node 68d811e61fd44e101337f122 INFO: 172.21.0.1:59494 - "POST /node/68d811e61fd44e101337f122 HTTP/1.0" 202 Accepted 2025-09-28 05:50:23,568 - lava_callback - INFO - Starting async job submit for node 68d811e61fd44e101337f122 2025-09-28 05:50:23,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:23,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811e61fd44e101337f122 HTTP/1.1" 200 None 2025-09-28 05:50:23,613 - lava_callback - DEBUG - Processing node 68d811e61fd44e101337f122: baseline-arm64 2025-09-28 05:50:23,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:50:24,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:50:24,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811e61fd44e101337f122/log.txt.gz 2025-09-28 05:50:24,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:50:24,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:50:24,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811e61fd44e101337f122/lava_callback.json.gz 2025-09-28 05:50:24,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:24,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811e61fd44e101337f122 HTTP/1.1" 200 None 2025-09-28 05:50:24,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:24,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d80b5e8dd6b39fd1aeb97b HTTP/1.1" 200 None 2025-09-28 05:50:24,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:50:24,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d811e61fd44e101337f122 HTTP/1.1" 200 None 2025-09-28 05:50:24,305 - lava_callback - INFO - Completed processing callback for node 68d811e61fd44e101337f122 INFO: 172.21.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48586 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 05:51:15,593 - lava_callback - INFO - Received callback for node 68d63392c1bc82a30b37adc2 INFO: 172.21.0.1:48596 - "POST /node/68d63392c1bc82a30b37adc2 HTTP/1.0" 202 Accepted 2025-09-28 05:51:15,600 - lava_callback - INFO - Starting async job submit for node 68d63392c1bc82a30b37adc2 2025-09-28 05:51:15,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:15,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63392c1bc82a30b37adc2 HTTP/1.1" 200 None 2025-09-28 05:51:15,652 - lava_callback - DEBUG - Processing node 68d63392c1bc82a30b37adc2: ltp-pty 2025-09-28 05:51:16,024 - lava_callback - ERROR - Error processing callback for node 68d63392c1bc82a30b37adc2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59624 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:51:28,976 - lava_callback - INFO - Received callback for node 68d811f41fd44e101337f150 INFO: 172.21.0.1:59630 - "POST /node/68d811f41fd44e101337f150 HTTP/1.0" 202 Accepted 2025-09-28 05:51:28,982 - lava_callback - INFO - Starting async job submit for node 68d811f41fd44e101337f150 2025-09-28 05:51:29,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:29,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811f41fd44e101337f150 HTTP/1.1" 200 None 2025-09-28 05:51:29,017 - lava_callback - DEBUG - Processing node 68d811f41fd44e101337f150: baseline-arm64 2025-09-28 05:51:29,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:51:29,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:51:29,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811f41fd44e101337f150/log.txt.gz 2025-09-28 05:51:29,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:51:29,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:51:29,627 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811f41fd44e101337f150/lava_callback.json.gz 2025-09-28 05:51:29,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:29,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811f41fd44e101337f150 HTTP/1.1" 200 None 2025-09-28 05:51:29,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:29,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-28 05:51:29,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:29,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d811f41fd44e101337f150 HTTP/1.1" 200 None 2025-09-28 05:51:29,733 - lava_callback - INFO - Completed processing callback for node 68d811f41fd44e101337f150 INFO: 172.21.0.1:56992 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:51:39,673 - lava_callback - INFO - Received callback for node 68d811f51fd44e101337f156 INFO: 172.21.0.1:37248 - "POST /node/68d811f51fd44e101337f156 HTTP/1.0" 202 Accepted 2025-09-28 05:51:39,679 - lava_callback - INFO - Starting async job submit for node 68d811f51fd44e101337f156 2025-09-28 05:51:39,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:39,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811f51fd44e101337f156 HTTP/1.1" 200 None 2025-09-28 05:51:39,714 - lava_callback - DEBUG - Processing node 68d811f51fd44e101337f156: baseline-arm64 2025-09-28 05:51:40,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:51:40,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:51:40,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811f51fd44e101337f156/log.txt.gz 2025-09-28 05:51:40,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:51:40,270 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:51:40,271 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811f51fd44e101337f156/lava_callback.json.gz 2025-09-28 05:51:40,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:40,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811f51fd44e101337f156 HTTP/1.1" 200 None 2025-09-28 05:51:40,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:40,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-28 05:51:40,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:51:40,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d811f51fd44e101337f156 HTTP/1.1" 200 None 2025-09-28 05:51:40,346 - lava_callback - INFO - Completed processing callback for node 68d811f51fd44e101337f156 INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:52:33,549 - lava_callback - INFO - Received callback for node 68d8c324a178a5bbc0ea6dee INFO: 172.21.0.1:48596 - "POST /node/68d8c324a178a5bbc0ea6dee HTTP/1.0" 202 Accepted 2025-09-28 05:52:33,556 - lava_callback - INFO - Starting async job submit for node 68d8c324a178a5bbc0ea6dee 2025-09-28 05:52:33,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:33,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c324a178a5bbc0ea6dee HTTP/1.1" 200 None 2025-09-28 05:52:33,634 - lava_callback - DEBUG - Processing node 68d8c324a178a5bbc0ea6dee: baseline-arm64 2025-09-28 05:52:34,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:34,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:34,893 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c324a178a5bbc0ea6dee/log.txt.gz 2025-09-28 05:52:34,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:35,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:35,123 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c324a178a5bbc0ea6dee/lava_callback.json.gz DEBUG: Setup failed for 68d8c324a178a5bbc0ea6dee. Transitting job node result: incomplete -> fail 2025-09-28 05:52:35,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:35,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c324a178a5bbc0ea6dee HTTP/1.1" 200 None 2025-09-28 05:52:35,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:35,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:52:35,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:35,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c324a178a5bbc0ea6dee HTTP/1.1" 200 None 2025-09-28 05:52:35,296 - lava_callback - INFO - Completed processing callback for node 68d8c324a178a5bbc0ea6dee INFO: 172.21.0.1:48598 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:52:40,295 - lava_callback - INFO - Received callback for node 68d811fa1fd44e101337f168 INFO: 172.21.0.1:46492 - "POST /node/68d811fa1fd44e101337f168 HTTP/1.0" 202 Accepted 2025-09-28 05:52:40,301 - lava_callback - INFO - Starting async job submit for node 68d811fa1fd44e101337f168 2025-09-28 05:52:40,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:40,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811fa1fd44e101337f168 HTTP/1.1" 200 None 2025-09-28 05:52:40,334 - lava_callback - DEBUG - Processing node 68d811fa1fd44e101337f168: baseline-arm64 2025-09-28 05:52:40,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:40,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:40,777 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811fa1fd44e101337f168/log.txt.gz 2025-09-28 05:52:40,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:40,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:40,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811fa1fd44e101337f168/lava_callback.json.gz 2025-09-28 05:52:40,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:40,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811fa1fd44e101337f168 HTTP/1.1" 200 None 2025-09-28 05:52:40,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:40,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-28 05:52:40,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:40,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d811fa1fd44e101337f168 HTTP/1.1" 200 None 2025-09-28 05:52:40,971 - lava_callback - INFO - Completed processing callback for node 68d811fa1fd44e101337f168 2025-09-28 05:52:44,904 - lava_callback - INFO - Received callback for node 68d63393c1bc82a30b37adca INFO: 172.21.0.1:46494 - "POST /node/68d63393c1bc82a30b37adca HTTP/1.0" 202 Accepted 2025-09-28 05:52:44,911 - lava_callback - INFO - Starting async job submit for node 68d63393c1bc82a30b37adca 2025-09-28 05:52:44,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:44,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63393c1bc82a30b37adca HTTP/1.1" 200 None 2025-09-28 05:52:44,976 - lava_callback - DEBUG - Processing node 68d63393c1bc82a30b37adca: ltp-smoketest 2025-09-28 05:52:45,427 - lava_callback - ERROR - Error processing callback for node 68d63393c1bc82a30b37adca: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46496 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:52:51,520 - lava_callback - INFO - Received callback for node 68d8c373a178a5bbc0ea6df7 INFO: 172.21.0.1:49266 - "POST /node/68d8c373a178a5bbc0ea6df7 HTTP/1.0" 202 Accepted 2025-09-28 05:52:51,527 - lava_callback - INFO - Starting async job submit for node 68d8c373a178a5bbc0ea6df7 2025-09-28 05:52:51,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:51,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c373a178a5bbc0ea6df7 HTTP/1.1" 200 None 2025-09-28 05:52:51,578 - lava_callback - DEBUG - Processing node 68d8c373a178a5bbc0ea6df7: baseline-arm64 2025-09-28 05:52:52,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:52,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:52,637 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c373a178a5bbc0ea6df7/log.txt.gz 2025-09-28 05:52:52,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:52,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:52,761 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c373a178a5bbc0ea6df7/lava_callback.json.gz DEBUG: Setup failed for 68d8c373a178a5bbc0ea6df7. Transitting job node result: incomplete -> fail 2025-09-28 05:52:52,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:52,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c373a178a5bbc0ea6df7 HTTP/1.1" 200 None 2025-09-28 05:52:52,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:52,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:52:52,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:52,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c373a178a5bbc0ea6df7 HTTP/1.1" 200 None 2025-09-28 05:52:52,892 - lava_callback - INFO - Completed processing callback for node 68d8c373a178a5bbc0ea6df7 INFO: 172.21.0.1:49270 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:52:59,035 - lava_callback - INFO - Received callback for node 68d811fb1fd44e101337f171 INFO: 172.21.0.1:49284 - "POST /node/68d811fb1fd44e101337f171 HTTP/1.0" 202 Accepted 2025-09-28 05:52:59,041 - lava_callback - INFO - Starting async job submit for node 68d811fb1fd44e101337f171 2025-09-28 05:52:59,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:59,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811fb1fd44e101337f171 HTTP/1.1" 200 None 2025-09-28 05:52:59,074 - lava_callback - DEBUG - Processing node 68d811fb1fd44e101337f171: baseline-arm64 2025-09-28 05:52:59,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:59,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:59,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811fb1fd44e101337f171/log.txt.gz 2025-09-28 05:52:59,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:52:59,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:52:59,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d811fb1fd44e101337f171/lava_callback.json.gz 2025-09-28 05:52:59,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:59,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d811fb1fd44e101337f171 HTTP/1.1" 200 None 2025-09-28 05:52:59,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:59,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64d12a5c3ab168ecc7a3d HTTP/1.1" 200 None 2025-09-28 05:52:59,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:52:59,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d811fb1fd44e101337f171 HTTP/1.1" 200 None 2025-09-28 05:52:59,672 - lava_callback - INFO - Completed processing callback for node 68d811fb1fd44e101337f171 INFO: 172.21.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50224 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:53:35,705 - lava_callback - INFO - Received callback for node 68d63393c1bc82a30b37adc5 INFO: 172.21.0.1:50212 - "POST /node/68d63393c1bc82a30b37adc5 HTTP/1.0" 202 Accepted 2025-09-28 05:53:35,711 - lava_callback - INFO - Starting async job submit for node 68d63393c1bc82a30b37adc5 2025-09-28 05:53:35,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:53:35,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63393c1bc82a30b37adc5 HTTP/1.1" 200 None 2025-09-28 05:53:35,786 - lava_callback - DEBUG - Processing node 68d63393c1bc82a30b37adc5: ltp-sched 2025-09-28 05:53:36,328 - lava_callback - ERROR - Error processing callback for node 68d63393c1bc82a30b37adc5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:50228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36332 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:55:46,163 - lava_callback - INFO - Received callback for node 68d8c45aa178a5bbc0ea6e09 INFO: 172.21.0.1:37522 - "POST /node/68d8c45aa178a5bbc0ea6e09 HTTP/1.0" 202 Accepted 2025-09-28 05:55:46,170 - lava_callback - INFO - Starting async job submit for node 68d8c45aa178a5bbc0ea6e09 2025-09-28 05:55:46,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:55:46,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c45aa178a5bbc0ea6e09 HTTP/1.1" 200 None 2025-09-28 05:55:46,223 - lava_callback - DEBUG - Processing node 68d8c45aa178a5bbc0ea6e09: baseline-arm64 INFO: 172.21.0.1:37538 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:55:47,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:55:47,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:55:47,448 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c45aa178a5bbc0ea6e09/log.txt.gz 2025-09-28 05:55:47,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:55:47,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:55:47,643 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c45aa178a5bbc0ea6e09/lava_callback.json.gz DEBUG: Setup failed for 68d8c45aa178a5bbc0ea6e09. Transitting job node result: incomplete -> fail 2025-09-28 05:55:47,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:55:47,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c45aa178a5bbc0ea6e09 HTTP/1.1" 200 None 2025-09-28 05:55:47,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:55:47,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:55:47,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:55:47,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c45aa178a5bbc0ea6e09 HTTP/1.1" 200 None 2025-09-28 05:55:47,951 - lava_callback - INFO - Completed processing callback for node 68d8c45aa178a5bbc0ea6e09 INFO: 172.21.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:57:11,641 - lava_callback - INFO - Received callback for node 68d63395c1bc82a30b37add7 INFO: 172.21.0.1:39286 - "POST /node/68d63395c1bc82a30b37add7 HTTP/1.0" 202 Accepted 2025-09-28 05:57:11,648 - lava_callback - INFO - Starting async job submit for node 68d63395c1bc82a30b37add7 2025-09-28 05:57:11,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:11,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63395c1bc82a30b37add7 HTTP/1.1" 200 None 2025-09-28 05:57:11,688 - lava_callback - DEBUG - Processing node 68d63395c1bc82a30b37add7: ltp-timers 2025-09-28 05:57:12,031 - lava_callback - ERROR - Error processing callback for node 68d63395c1bc82a30b37add7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39296 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 05:57:16,050 - lava_callback - INFO - Received callback for node 68d8c48ca178a5bbc0ea6e12 INFO: 172.21.0.1:39308 - "POST /node/68d8c48ca178a5bbc0ea6e12 HTTP/1.0" 202 Accepted 2025-09-28 05:57:16,057 - lava_callback - INFO - Starting async job submit for node 68d8c48ca178a5bbc0ea6e12 2025-09-28 05:57:16,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:16,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48ca178a5bbc0ea6e12 HTTP/1.1" 200 None 2025-09-28 05:57:16,116 - lava_callback - DEBUG - Processing node 68d8c48ca178a5bbc0ea6e12: baseline-arm64 INFO: 172.21.0.1:39312 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:57:17,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:57:17,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:57:17,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48ca178a5bbc0ea6e12/log.txt.gz 2025-09-28 05:57:17,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:57:17,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:57:17,461 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48ca178a5bbc0ea6e12/lava_callback.json.gz DEBUG: Setup failed for 68d8c48ca178a5bbc0ea6e12. Transitting job node result: incomplete -> fail 2025-09-28 05:57:17,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:17,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48ca178a5bbc0ea6e12 HTTP/1.1" 200 None 2025-09-28 05:57:17,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:17,566 - lava_callback - INFO - Received callback for node 68d8c45ba178a5bbc0ea6e0c INFO: 172.21.0.1:39316 - "POST /node/68d8c45ba178a5bbc0ea6e0c HTTP/1.0" 202 Accepted 2025-09-28 05:57:17,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:57:17,574 - lava_callback - INFO - Starting async job submit for node 68d8c45ba178a5bbc0ea6e0c 2025-09-28 05:57:17,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:17,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:17,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c48ca178a5bbc0ea6e12 HTTP/1.1" 200 None 2025-09-28 05:57:17,646 - lava_callback - INFO - Completed processing callback for node 68d8c48ca178a5bbc0ea6e12 2025-09-28 05:57:17,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c45ba178a5bbc0ea6e0c HTTP/1.1" 200 None 2025-09-28 05:57:17,697 - lava_callback - DEBUG - Processing node 68d8c45ba178a5bbc0ea6e0c: baseline-arm64 2025-09-28 05:57:18,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:57:18,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:57:18,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c45ba178a5bbc0ea6e0c/log.txt.gz 2025-09-28 05:57:18,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:57:19,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:57:19,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c45ba178a5bbc0ea6e0c/lava_callback.json.gz DEBUG: Setup failed for 68d8c45ba178a5bbc0ea6e0c. Transitting job node result: incomplete -> fail 2025-09-28 05:57:19,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:19,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c45ba178a5bbc0ea6e0c HTTP/1.1" 200 None 2025-09-28 05:57:19,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:19,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:57:19,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:19,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c45ba178a5bbc0ea6e0c HTTP/1.1" 200 None 2025-09-28 05:57:19,285 - lava_callback - INFO - Completed processing callback for node 68d8c45ba178a5bbc0ea6e0c INFO: 172.21.0.1:38428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49366 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:57:55,814 - lava_callback - INFO - Received callback for node 68d63395c1bc82a30b37adda INFO: 172.21.0.1:60234 - "POST /node/68d63395c1bc82a30b37adda HTTP/1.0" 202 Accepted 2025-09-28 05:57:55,821 - lava_callback - INFO - Starting async job submit for node 68d63395c1bc82a30b37adda 2025-09-28 05:57:55,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:57:55,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63395c1bc82a30b37adda HTTP/1.1" 200 None 2025-09-28 05:57:55,871 - lava_callback - DEBUG - Processing node 68d63395c1bc82a30b37adda: ltp-watchqueue 2025-09-28 05:57:56,257 - lava_callback - ERROR - Error processing callback for node 68d63395c1bc82a30b37adda: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59626 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:59:25,584 - lava_callback - INFO - Received callback for node 68d633a5c1bc82a30b37ae07 INFO: 172.21.0.1:53268 - "POST /node/68d633a5c1bc82a30b37ae07 HTTP/1.0" 202 Accepted 2025-09-28 05:59:25,590 - lava_callback - INFO - Starting async job submit for node 68d633a5c1bc82a30b37ae07 2025-09-28 05:59:25,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:59:25,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633a5c1bc82a30b37ae07 HTTP/1.1" 200 None 2025-09-28 05:59:25,647 - lava_callback - DEBUG - Processing node 68d633a5c1bc82a30b37ae07: baseline-arm-broonie 2025-09-28 05:59:25,861 - lava_callback - ERROR - Error processing callback for node 68d633a5c1bc82a30b37ae07: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58976 - "GET / HTTP/1.0" 200 OK 2025-09-28 05:59:31,254 - lava_callback - INFO - Received callback for node 68d8c38ca178a5bbc0ea6dfd INFO: 172.21.0.1:58978 - "POST /node/68d8c38ca178a5bbc0ea6dfd HTTP/1.0" 202 Accepted 2025-09-28 05:59:31,261 - lava_callback - INFO - Starting async job submit for node 68d8c38ca178a5bbc0ea6dfd 2025-09-28 05:59:31,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:59:31,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c38ca178a5bbc0ea6dfd HTTP/1.1" 200 None 2025-09-28 05:59:31,332 - lava_callback - DEBUG - Processing node 68d8c38ca178a5bbc0ea6dfd: baseline-arm64 2025-09-28 05:59:32,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:59:32,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:59:32,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c38ca178a5bbc0ea6dfd/log.txt.gz 2025-09-28 05:59:32,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 05:59:32,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 05:59:32,639 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c38ca178a5bbc0ea6dfd/lava_callback.json.gz DEBUG: Setup failed for 68d8c38ca178a5bbc0ea6dfd. Transitting job node result: incomplete -> fail 2025-09-28 05:59:32,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:59:32,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c38ca178a5bbc0ea6dfd HTTP/1.1" 200 None 2025-09-28 05:59:32,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:59:32,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 05:59:32,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 05:59:32,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c38ca178a5bbc0ea6dfd HTTP/1.1" 200 None 2025-09-28 05:59:32,775 - lava_callback - INFO - Completed processing callback for node 68d8c38ca178a5bbc0ea6dfd INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36360 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:01:08,976 - lava_callback - INFO - Received callback for node 68d8c64ca178a5bbc0ea6e42 INFO: 172.21.0.1:36364 - "POST /node/68d8c64ca178a5bbc0ea6e42 HTTP/1.0" 202 Accepted 2025-09-28 06:01:08,985 - lava_callback - INFO - Starting async job submit for node 68d8c64ca178a5bbc0ea6e42 2025-09-28 06:01:09,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:01:09,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c64ca178a5bbc0ea6e42 HTTP/1.1" 200 None 2025-09-28 06:01:09,025 - lava_callback - DEBUG - Processing node 68d8c64ca178a5bbc0ea6e42: baseline-arm64-broonie 2025-09-28 06:01:09,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:01:09,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:01:09,411 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8c64ca178a5bbc0ea6e42/log.txt.gz 2025-09-28 06:01:09,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:01:09,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:01:09,550 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8c64ca178a5bbc0ea6e42/lava_callback.json.gz 2025-09-28 06:01:09,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:01:09,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c64ca178a5bbc0ea6e42 HTTP/1.1" 200 None 2025-09-28 06:01:09,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:01:09,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:01:09,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:01:09,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c64ca178a5bbc0ea6e42 HTTP/1.1" 200 None 2025-09-28 06:01:09,638 - lava_callback - INFO - Completed processing callback for node 68d8c64ca178a5bbc0ea6e42 INFO: 172.21.0.1:44046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41864 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:02:01,926 - lava_callback - INFO - Received callback for node 68d8c38da178a5bbc0ea6e00 INFO: 172.21.0.1:49610 - "POST /node/68d8c38da178a5bbc0ea6e00 HTTP/1.0" 202 Accepted 2025-09-28 06:02:01,933 - lava_callback - INFO - Starting async job submit for node 68d8c38da178a5bbc0ea6e00 2025-09-28 06:02:01,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:01,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c38da178a5bbc0ea6e00 HTTP/1.1" 200 None 2025-09-28 06:02:01,986 - lava_callback - DEBUG - Processing node 68d8c38da178a5bbc0ea6e00: baseline-arm64 2025-09-28 06:02:02,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:02:03,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:02:03,082 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c38da178a5bbc0ea6e00/log.txt.gz 2025-09-28 06:02:03,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:02:03,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:02:03,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c38da178a5bbc0ea6e00/lava_callback.json.gz DEBUG: Setup failed for 68d8c38da178a5bbc0ea6e00. Transitting job node result: incomplete -> fail 2025-09-28 06:02:03,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:03,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c38da178a5bbc0ea6e00 HTTP/1.1" 200 None 2025-09-28 06:02:03,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:03,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:02:03,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:03,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c38da178a5bbc0ea6e00 HTTP/1.1" 200 None 2025-09-28 06:02:03,410 - lava_callback - INFO - Completed processing callback for node 68d8c38da178a5bbc0ea6e00 INFO: 172.21.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53944 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 06:02:16,053 - lava_callback - INFO - Received callback for node 68d633b5c1bc82a30b37ae22 INFO: 172.21.0.1:53950 - "POST /node/68d633b5c1bc82a30b37ae22 HTTP/1.0" 202 Accepted 2025-09-28 06:02:16,059 - lava_callback - INFO - Starting async job submit for node 68d633b5c1bc82a30b37ae22 2025-09-28 06:02:16,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:16,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633b5c1bc82a30b37ae22 HTTP/1.1" 200 None 2025-09-28 06:02:16,107 - lava_callback - DEBUG - Processing node 68d633b5c1bc82a30b37ae22: ltp-cap-bounds 2025-09-28 06:02:16,455 - lava_callback - ERROR - Error processing callback for node 68d633b5c1bc82a30b37ae22: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:02:24,125 - lava_callback - INFO - Received callback for node 68d8b863a178a5bbc0ea6bc0 INFO: 172.21.0.1:58124 - "POST /node/68d8b863a178a5bbc0ea6bc0 HTTP/1.0" 202 Accepted 2025-09-28 06:02:24,131 - lava_callback - INFO - Starting async job submit for node 68d8b863a178a5bbc0ea6bc0 2025-09-28 06:02:24,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:24,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b863a178a5bbc0ea6bc0 HTTP/1.1" 200 None 2025-09-28 06:02:24,173 - lava_callback - DEBUG - Processing node 68d8b863a178a5bbc0ea6bc0: baseline-arm 2025-09-28 06:02:24,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:02:24,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:02:24,587 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b863a178a5bbc0ea6bc0/log.txt.gz 2025-09-28 06:02:24,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:02:24,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:02:24,741 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b863a178a5bbc0ea6bc0/lava_callback.json.gz DEBUG: Setup failed for 68d8b863a178a5bbc0ea6bc0. Transitting job node result: incomplete -> fail 2025-09-28 06:02:24,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:24,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b863a178a5bbc0ea6bc0 HTTP/1.1" 200 None 2025-09-28 06:02:24,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:24,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 06:02:24,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:24,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b863a178a5bbc0ea6bc0 HTTP/1.1" 200 None 2025-09-28 06:02:24,873 - lava_callback - INFO - Completed processing callback for node 68d8b863a178a5bbc0ea6bc0 INFO: 172.21.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35210 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:02:46,900 - lava_callback - INFO - Received callback for node 68d8034c8dd6b39fd1aeb8ee INFO: 172.21.0.1:35212 - "POST /node/68d8034c8dd6b39fd1aeb8ee HTTP/1.0" 202 Accepted 2025-09-28 06:02:46,908 - lava_callback - INFO - Starting async job submit for node 68d8034c8dd6b39fd1aeb8ee 2025-09-28 06:02:46,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:46,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8034c8dd6b39fd1aeb8ee HTTP/1.1" 200 None 2025-09-28 06:02:46,963 - lava_callback - DEBUG - Processing node 68d8034c8dd6b39fd1aeb8ee: baseline-arm64 2025-09-28 06:02:47,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:02:48,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:02:48,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8034c8dd6b39fd1aeb8ee/log.txt.gz 2025-09-28 06:02:48,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:02:48,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:02:48,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8034c8dd6b39fd1aeb8ee/lava_callback.json.gz DEBUG: Setup failed for 68d8034c8dd6b39fd1aeb8ee. Transitting job node result: incomplete -> fail 2025-09-28 06:02:48,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:48,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8034c8dd6b39fd1aeb8ee HTTP/1.1" 200 None 2025-09-28 06:02:48,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:48,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:02:48,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:02:48,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8034c8dd6b39fd1aeb8ee HTTP/1.1" 200 None 2025-09-28 06:02:48,364 - lava_callback - INFO - Completed processing callback for node 68d8034c8dd6b39fd1aeb8ee INFO: 172.21.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37048 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:03:22,794 - lava_callback - INFO - Received callback for node 68d8c549a178a5bbc0ea6e2f INFO: 172.21.0.1:41232 - "POST /node/68d8c549a178a5bbc0ea6e2f HTTP/1.0" 202 Accepted 2025-09-28 06:03:22,801 - lava_callback - INFO - Starting async job submit for node 68d8c549a178a5bbc0ea6e2f 2025-09-28 06:03:22,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:03:22,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c549a178a5bbc0ea6e2f HTTP/1.1" 200 None 2025-09-28 06:03:22,851 - lava_callback - DEBUG - Processing node 68d8c549a178a5bbc0ea6e2f: baseline-arm64 2025-09-28 06:03:23,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:03:23,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:03:23,943 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c549a178a5bbc0ea6e2f/log.txt.gz 2025-09-28 06:03:23,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:03:24,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:03:24,124 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c549a178a5bbc0ea6e2f/lava_callback.json.gz DEBUG: Setup failed for 68d8c549a178a5bbc0ea6e2f. Transitting job node result: incomplete -> fail 2025-09-28 06:03:24,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:03:24,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c549a178a5bbc0ea6e2f HTTP/1.1" 200 None 2025-09-28 06:03:24,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:03:24,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:03:24,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:03:24,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c549a178a5bbc0ea6e2f HTTP/1.1" 200 None 2025-09-28 06:03:24,257 - lava_callback - INFO - Completed processing callback for node 68d8c549a178a5bbc0ea6e2f INFO: 172.21.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56522 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:06:05,070 - lava_callback - INFO - Received callback for node 68d8c48ca178a5bbc0ea6e15 INFO: 172.21.0.1:48726 - "POST /node/68d8c48ca178a5bbc0ea6e15 HTTP/1.0" 202 Accepted 2025-09-28 06:06:05,082 - lava_callback - INFO - Starting async job submit for node 68d8c48ca178a5bbc0ea6e15 2025-09-28 06:06:05,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:06:05,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48ca178a5bbc0ea6e15 HTTP/1.1" 200 None 2025-09-28 06:06:05,165 - lava_callback - DEBUG - Processing node 68d8c48ca178a5bbc0ea6e15: baseline-arm64 2025-09-28 06:06:05,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:06:05,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:06:05,945 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48ca178a5bbc0ea6e15/log.txt.gz 2025-09-28 06:06:05,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:06:06,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:06:06,080 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48ca178a5bbc0ea6e15/lava_callback.json.gz DEBUG: Setup failed for 68d8c48ca178a5bbc0ea6e15. Transitting job node result: incomplete -> fail 2025-09-28 06:06:06,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:06:06,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48ca178a5bbc0ea6e15 HTTP/1.1" 200 None 2025-09-28 06:06:06,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:06:06,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:06:06,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:06:06,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c48ca178a5bbc0ea6e15 HTTP/1.1" 200 None 2025-09-28 06:06:06,253 - lava_callback - INFO - Completed processing callback for node 68d8c48ca178a5bbc0ea6e15 INFO: 172.21.0.1:48738 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:06:07,949 - lava_callback - INFO - Received callback for node 68d633b7c1bc82a30b37ae25 INFO: 172.21.0.1:48740 - "POST /node/68d633b7c1bc82a30b37ae25 HTTP/1.0" 202 Accepted 2025-09-28 06:06:07,957 - lava_callback - INFO - Starting async job submit for node 68d633b7c1bc82a30b37ae25 2025-09-28 06:06:08,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:06:08,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633b7c1bc82a30b37ae25 HTTP/1.1" 200 None 2025-09-28 06:06:08,078 - lava_callback - DEBUG - Processing node 68d633b7c1bc82a30b37ae25: ltp-containers 2025-09-28 06:06:08,883 - lava_callback - ERROR - Error processing callback for node 68d633b7c1bc82a30b37ae25: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55664 - "GET /node/68d4ff79ef0c6c1c70206455%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51882 - "GET /node/68d5192ab70fb2a1e20f11fa%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51888 - "GET /node/68d4fe7bef0c6c1c702061c5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51896 - "GET /node/68d4fcbfef0c6c1c70206058%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51916 - "GET /node/68d4cb2fa5c2c05277bd061b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51924 - "GET /node/68d4fe80ef0c6c1c702061cb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48334 - "GET /node/68d51dd9b70fb2a1e20f1868%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48338 - "GET /node/68d501acef0c6c1c70206576%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48350 - "GET /node/68d51dd6b70fb2a1e20f1865%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48358 - "GET /node/68d4cfe1a5c2c05277bd0b31%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48362 - "GET /node/68d51928b70fb2a1e20f11f7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:10:16,531 - lava_callback - INFO - Received callback for node 68d63394c1bc82a30b37add4 INFO: 172.21.0.1:43758 - "POST /node/68d63394c1bc82a30b37add4 HTTP/1.0" 202 Accepted 2025-09-28 06:10:16,538 - lava_callback - INFO - Starting async job submit for node 68d63394c1bc82a30b37add4 2025-09-28 06:10:16,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:10:16,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63394c1bc82a30b37add4 HTTP/1.1" 200 None 2025-09-28 06:10:16,580 - lava_callback - DEBUG - Processing node 68d63394c1bc82a30b37add4: ltp-syscalls-ipc 2025-09-28 06:10:17,454 - lava_callback - ERROR - Error processing callback for node 68d63394c1bc82a30b37add4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:13:14,446 - lava_callback - INFO - Received callback for node 68d8c54aa178a5bbc0ea6e32 INFO: 172.21.0.1:54566 - "POST /node/68d8c54aa178a5bbc0ea6e32 HTTP/1.0" 202 Accepted 2025-09-28 06:13:14,453 - lava_callback - INFO - Starting async job submit for node 68d8c54aa178a5bbc0ea6e32 2025-09-28 06:13:14,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:14,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c54aa178a5bbc0ea6e32 HTTP/1.1" 200 None 2025-09-28 06:13:14,525 - lava_callback - DEBUG - Processing node 68d8c54aa178a5bbc0ea6e32: baseline-arm64 INFO: 172.21.0.1:54574 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 06:13:15,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:13:15,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:13:15,843 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c54aa178a5bbc0ea6e32/log.txt.gz 2025-09-28 06:13:15,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:13:16,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:13:16,042 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c54aa178a5bbc0ea6e32/lava_callback.json.gz DEBUG: Setup failed for 68d8c54aa178a5bbc0ea6e32. Transitting job node result: incomplete -> fail 2025-09-28 06:13:16,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:16,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c54aa178a5bbc0ea6e32 HTTP/1.1" 200 None 2025-09-28 06:13:16,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:16,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:13:16,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:16,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c54aa178a5bbc0ea6e32 HTTP/1.1" 200 None 2025-09-28 06:13:16,178 - lava_callback - INFO - Completed processing callback for node 68d8c54aa178a5bbc0ea6e32 INFO: 172.21.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42920 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:13:50,435 - lava_callback - INFO - Received callback for node 68d8c67fa178a5bbc0ea6e4e INFO: 172.21.0.1:41052 - "POST /node/68d8c67fa178a5bbc0ea6e4e HTTP/1.0" 202 Accepted 2025-09-28 06:13:50,443 - lava_callback - INFO - Starting async job submit for node 68d8c67fa178a5bbc0ea6e4e 2025-09-28 06:13:50,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:50,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c67fa178a5bbc0ea6e4e HTTP/1.1" 200 None 2025-09-28 06:13:50,493 - lava_callback - DEBUG - Processing node 68d8c67fa178a5bbc0ea6e4e: baseline-arm64 2025-09-28 06:13:51,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:13:51,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:13:51,741 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c67fa178a5bbc0ea6e4e/log.txt.gz 2025-09-28 06:13:51,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:13:51,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:13:51,913 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c67fa178a5bbc0ea6e4e/lava_callback.json.gz DEBUG: Setup failed for 68d8c67fa178a5bbc0ea6e4e. Transitting job node result: incomplete -> fail 2025-09-28 06:13:51,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:51,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c67fa178a5bbc0ea6e4e HTTP/1.1" 200 None 2025-09-28 06:13:51,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:52,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:13:52,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:13:52,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c67fa178a5bbc0ea6e4e HTTP/1.1" 200 None 2025-09-28 06:13:52,040 - lava_callback - INFO - Completed processing callback for node 68d8c67fa178a5bbc0ea6e4e INFO: 172.21.0.1:41054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59490 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:14:35,026 - lava_callback - INFO - Received callback for node 68d8c67ea178a5bbc0ea6e4b INFO: 172.21.0.1:59504 - "POST /node/68d8c67ea178a5bbc0ea6e4b HTTP/1.0" 202 Accepted 2025-09-28 06:14:35,034 - lava_callback - INFO - Starting async job submit for node 68d8c67ea178a5bbc0ea6e4b 2025-09-28 06:14:35,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:14:35,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c67ea178a5bbc0ea6e4b HTTP/1.1" 200 None 2025-09-28 06:14:35,109 - lava_callback - DEBUG - Processing node 68d8c67ea178a5bbc0ea6e4b: baseline-arm64 2025-09-28 06:14:36,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:14:36,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:14:36,428 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c67ea178a5bbc0ea6e4b/log.txt.gz 2025-09-28 06:14:36,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59520 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:14:36,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:14:36,633 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c67ea178a5bbc0ea6e4b/lava_callback.json.gz DEBUG: Setup failed for 68d8c67ea178a5bbc0ea6e4b. Transitting job node result: incomplete -> fail 2025-09-28 06:14:36,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:14:36,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c67ea178a5bbc0ea6e4b HTTP/1.1" 200 None 2025-09-28 06:14:36,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:14:36,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:14:36,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:14:36,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c67ea178a5bbc0ea6e4b HTTP/1.1" 200 None 2025-09-28 06:14:36,775 - lava_callback - INFO - Completed processing callback for node 68d8c67ea178a5bbc0ea6e4b INFO: 172.21.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:15:54,827 - lava_callback - INFO - Received callback for node 68d8166c1fd44e101337f5c3 INFO: 172.21.0.1:43232 - "POST /node/68d8166c1fd44e101337f5c3 HTTP/1.0" 202 Accepted 2025-09-28 06:15:54,834 - lava_callback - INFO - Starting async job submit for node 68d8166c1fd44e101337f5c3 2025-09-28 06:15:54,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:15:54,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8166c1fd44e101337f5c3 HTTP/1.1" 200 None 2025-09-28 06:15:54,909 - lava_callback - DEBUG - Processing node 68d8166c1fd44e101337f5c3: baseline-arm64 2025-09-28 06:15:55,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:15:55,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:15:55,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8166c1fd44e101337f5c3/log.txt.gz 2025-09-28 06:15:55,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:15:56,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:15:56,165 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8166c1fd44e101337f5c3/lava_callback.json.gz DEBUG: Setup failed for 68d8166c1fd44e101337f5c3. Transitting job node result: incomplete -> fail 2025-09-28 06:15:56,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:15:56,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8166c1fd44e101337f5c3 HTTP/1.1" 200 None 2025-09-28 06:15:56,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:15:56,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:15:56,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:15:56,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8166c1fd44e101337f5c3 HTTP/1.1" 200 None 2025-09-28 06:15:56,300 - lava_callback - INFO - Completed processing callback for node 68d8166c1fd44e101337f5c3 INFO: 172.21.0.1:43246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38404 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:16:06,699 - lava_callback - INFO - Received callback for node 68d8c48fa178a5bbc0ea6e1b INFO: 172.21.0.1:38420 - "POST /node/68d8c48fa178a5bbc0ea6e1b HTTP/1.0" 202 Accepted 2025-09-28 06:16:06,707 - lava_callback - INFO - Starting async job submit for node 68d8c48fa178a5bbc0ea6e1b 2025-09-28 06:16:06,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:06,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48fa178a5bbc0ea6e1b HTTP/1.1" 200 None 2025-09-28 06:16:06,766 - lava_callback - DEBUG - Processing node 68d8c48fa178a5bbc0ea6e1b: baseline-arm64 2025-09-28 06:16:07,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:16:08,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:16:08,003 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48fa178a5bbc0ea6e1b/log.txt.gz 2025-09-28 06:16:08,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:16:08,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:16:08,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48fa178a5bbc0ea6e1b/lava_callback.json.gz DEBUG: Setup failed for 68d8c48fa178a5bbc0ea6e1b. Transitting job node result: incomplete -> fail 2025-09-28 06:16:08,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:08,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48fa178a5bbc0ea6e1b HTTP/1.1" 200 None 2025-09-28 06:16:08,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:08,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:16:08,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:08,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c48fa178a5bbc0ea6e1b HTTP/1.1" 200 None 2025-09-28 06:16:08,363 - lava_callback - INFO - Completed processing callback for node 68d8c48fa178a5bbc0ea6e1b INFO: 172.21.0.1:55238 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:16:14,535 - lava_callback - INFO - Received callback for node 68d8166d1fd44e101337f5c6 INFO: 172.21.0.1:55240 - "POST /node/68d8166d1fd44e101337f5c6 HTTP/1.0" 202 Accepted 2025-09-28 06:16:14,542 - lava_callback - INFO - Starting async job submit for node 68d8166d1fd44e101337f5c6 2025-09-28 06:16:14,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:14,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8166d1fd44e101337f5c6 HTTP/1.1" 200 None 2025-09-28 06:16:14,600 - lava_callback - DEBUG - Processing node 68d8166d1fd44e101337f5c6: baseline-arm64 INFO: 172.21.0.1:55252 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 06:16:15,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:16:15,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:16:15,701 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8166d1fd44e101337f5c6/log.txt.gz 2025-09-28 06:16:15,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:16:15,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:16:15,876 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8166d1fd44e101337f5c6/lava_callback.json.gz DEBUG: Setup failed for 68d8166d1fd44e101337f5c6. Transitting job node result: incomplete -> fail 2025-09-28 06:16:15,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:15,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8166d1fd44e101337f5c6 HTTP/1.1" 200 None 2025-09-28 06:16:15,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:15,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:16:15,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:16:16,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8166d1fd44e101337f5c6 HTTP/1.1" 200 None 2025-09-28 06:16:16,011 - lava_callback - INFO - Completed processing callback for node 68d8166d1fd44e101337f5c6 INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57140 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:17:36,775 - lava_callback - INFO - Received callback for node 68d633bec1bc82a30b37ae31 INFO: 172.21.0.1:57150 - "POST /node/68d633bec1bc82a30b37ae31 HTTP/1.0" 202 Accepted 2025-09-28 06:17:36,782 - lava_callback - INFO - Starting async job submit for node 68d633bec1bc82a30b37ae31 2025-09-28 06:17:36,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:36,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633bec1bc82a30b37ae31 HTTP/1.1" 200 None 2025-09-28 06:17:36,915 - lava_callback - DEBUG - Processing node 68d633bec1bc82a30b37ae31: ltp-fs-bind 2025-09-28 06:17:37,417 - lava_callback - ERROR - Error processing callback for node 68d633bec1bc82a30b37ae31: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:60156 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:17:56,432 - lava_callback - INFO - Received callback for node 68d8c533a178a5bbc0ea6e26 INFO: 172.21.0.1:43996 - "POST /node/68d8c533a178a5bbc0ea6e26 HTTP/1.0" 202 Accepted 2025-09-28 06:17:56,439 - lava_callback - INFO - Starting async job submit for node 68d8c533a178a5bbc0ea6e26 2025-09-28 06:17:56,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:56,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c533a178a5bbc0ea6e26 HTTP/1.1" 200 None 2025-09-28 06:17:56,495 - lava_callback - DEBUG - Processing node 68d8c533a178a5bbc0ea6e26: baseline-arm64 INFO: 172.21.0.1:44002 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:17:57,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:17:57,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:17:57,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c533a178a5bbc0ea6e26/log.txt.gz 2025-09-28 06:17:57,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:17:57,905 - lava_callback - INFO - Received callback for node 68d8c48fa178a5bbc0ea6e1e INFO: 172.21.0.1:44004 - "POST /node/68d8c48fa178a5bbc0ea6e1e HTTP/1.0" 202 Accepted 2025-09-28 06:17:57,913 - lava_callback - INFO - Starting async job submit for node 68d8c48fa178a5bbc0ea6e1e 2025-09-28 06:17:57,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:57,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:17:57,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c533a178a5bbc0ea6e26/lava_callback.json.gz 2025-09-28 06:17:58,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48fa178a5bbc0ea6e1e HTTP/1.1" 200 None 2025-09-28 06:17:58,033 - lava_callback - DEBUG - Processing node 68d8c48fa178a5bbc0ea6e1e: baseline-arm64 DEBUG: Setup failed for 68d8c533a178a5bbc0ea6e26. Transitting job node result: incomplete -> fail 2025-09-28 06:17:58,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:58,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c533a178a5bbc0ea6e26 HTTP/1.1" 200 None 2025-09-28 06:17:58,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:58,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:17:58,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:58,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c533a178a5bbc0ea6e26 HTTP/1.1" 200 None 2025-09-28 06:17:58,324 - lava_callback - INFO - Completed processing callback for node 68d8c533a178a5bbc0ea6e26 2025-09-28 06:17:59,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:17:59,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:17:59,382 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48fa178a5bbc0ea6e1e/log.txt.gz 2025-09-28 06:17:59,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:17:59,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:17:59,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c48fa178a5bbc0ea6e1e/lava_callback.json.gz DEBUG: Setup failed for 68d8c48fa178a5bbc0ea6e1e. Transitting job node result: incomplete -> fail 2025-09-28 06:17:59,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:59,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c48fa178a5bbc0ea6e1e HTTP/1.1" 200 None 2025-09-28 06:17:59,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:59,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:17:59,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:17:59,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c48fa178a5bbc0ea6e1e HTTP/1.1" 200 None 2025-09-28 06:17:59,742 - lava_callback - INFO - Completed processing callback for node 68d8c48fa178a5bbc0ea6e1e INFO: 172.21.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:18:57,340 - lava_callback - INFO - Received callback for node 68d633b9c1bc82a30b37ae28 INFO: 172.21.0.1:39876 - "POST /node/68d633b9c1bc82a30b37ae28 HTTP/1.0" 202 Accepted 2025-09-28 06:18:57,351 - lava_callback - INFO - Starting async job submit for node 68d633b9c1bc82a30b37ae28 2025-09-28 06:18:57,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:18:57,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633b9c1bc82a30b37ae28 HTTP/1.1" 200 None 2025-09-28 06:18:57,761 - lava_callback - DEBUG - Processing node 68d633b9c1bc82a30b37ae28: ltp-controllers 2025-09-28 06:19:00,623 - lava_callback - ERROR - Error processing callback for node 68d633b9c1bc82a30b37ae28: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52440 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:21:56,807 - lava_callback - INFO - Received callback for node 68d633bfc1bc82a30b37ae34 INFO: 172.21.0.1:52452 - "POST /node/68d633bfc1bc82a30b37ae34 HTTP/1.0" 202 Accepted 2025-09-28 06:21:56,813 - lava_callback - INFO - Starting async job submit for node 68d633bfc1bc82a30b37ae34 2025-09-28 06:21:56,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:21:56,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633bfc1bc82a30b37ae34 HTTP/1.1" 200 None 2025-09-28 06:21:56,852 - lava_callback - DEBUG - Processing node 68d633bfc1bc82a30b37ae34: ltp-fsx 2025-09-28 06:21:57,213 - lava_callback - ERROR - Error processing callback for node 68d633bfc1bc82a30b37ae34: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-28 06:22:04,399 - lava_callback - INFO - Received callback for node 68d8b864a178a5bbc0ea6bc3 INFO: 172.21.0.1:44850 - "POST /node/68d8b864a178a5bbc0ea6bc3 HTTP/1.0" 202 Accepted 2025-09-28 06:22:04,405 - lava_callback - INFO - Starting async job submit for node 68d8b864a178a5bbc0ea6bc3 2025-09-28 06:22:04,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:22:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b864a178a5bbc0ea6bc3 HTTP/1.1" 200 None 2025-09-28 06:22:04,468 - lava_callback - DEBUG - Processing node 68d8b864a178a5bbc0ea6bc3: baseline-arm 2025-09-28 06:22:04,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:22:04,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:22:04,881 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b864a178a5bbc0ea6bc3/log.txt.gz 2025-09-28 06:22:04,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:22:04,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:22:04,998 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8b864a178a5bbc0ea6bc3/lava_callback.json.gz DEBUG: Setup failed for 68d8b864a178a5bbc0ea6bc3. Transitting job node result: incomplete -> fail 2025-09-28 06:22:05,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:22:05,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b864a178a5bbc0ea6bc3 HTTP/1.1" 200 None 2025-09-28 06:22:05,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:22:05,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 06:22:05,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:22:05,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b864a178a5bbc0ea6bc3 HTTP/1.1" 200 None 2025-09-28 06:22:05,111 - lava_callback - INFO - Completed processing callback for node 68d8b864a178a5bbc0ea6bc3 INFO: 172.21.0.1:44852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:23:05,892 - lava_callback - INFO - Received callback for node 68d816721fd44e101337f5cd INFO: 172.21.0.1:37068 - "POST /node/68d816721fd44e101337f5cd HTTP/1.0" 202 Accepted 2025-09-28 06:23:05,900 - lava_callback - INFO - Starting async job submit for node 68d816721fd44e101337f5cd 2025-09-28 06:23:05,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:23:05,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816721fd44e101337f5cd HTTP/1.1" 200 None 2025-09-28 06:23:05,954 - lava_callback - DEBUG - Processing node 68d816721fd44e101337f5cd: baseline-arm64 INFO: 172.21.0.1:37074 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:23:06,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:23:07,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:23:07,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816721fd44e101337f5cd/log.txt.gz 2025-09-28 06:23:07,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:23:07,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:23:07,308 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816721fd44e101337f5cd/lava_callback.json.gz DEBUG: Setup failed for 68d816721fd44e101337f5cd. Transitting job node result: incomplete -> fail 2025-09-28 06:23:07,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:23:07,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816721fd44e101337f5cd HTTP/1.1" 200 None 2025-09-28 06:23:07,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:23:07,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:23:07,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:23:07,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816721fd44e101337f5cd HTTP/1.1" 200 None 2025-09-28 06:23:07,483 - lava_callback - INFO - Completed processing callback for node 68d816721fd44e101337f5cd INFO: 172.21.0.1:40226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40240 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:23:16,877 - lava_callback - INFO - Received callback for node 68d633c1c1bc82a30b37ae37 INFO: 172.21.0.1:40254 - "POST /node/68d633c1c1bc82a30b37ae37 HTTP/1.0" 202 Accepted 2025-09-28 06:23:16,884 - lava_callback - INFO - Starting async job submit for node 68d633c1c1bc82a30b37ae37 2025-09-28 06:23:16,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:23:16,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633c1c1bc82a30b37ae37 HTTP/1.1" 200 None 2025-09-28 06:23:16,928 - lava_callback - DEBUG - Processing node 68d633c1c1bc82a30b37ae37: ltp-input 2025-09-28 06:23:17,321 - lava_callback - ERROR - Error processing callback for node 68d633c1c1bc82a30b37ae37: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58360 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:26:17,032 - lava_callback - INFO - Received callback for node 68d633c3c1bc82a30b37ae3a INFO: 172.21.0.1:58372 - "POST /node/68d633c3c1bc82a30b37ae3a HTTP/1.0" 202 Accepted 2025-09-28 06:26:17,039 - lava_callback - INFO - Starting async job submit for node 68d633c3c1bc82a30b37ae3a 2025-09-28 06:26:17,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:26:17,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633c3c1bc82a30b37ae3a HTTP/1.1" 200 None 2025-09-28 06:26:17,083 - lava_callback - DEBUG - Processing node 68d633c3c1bc82a30b37ae3a: ltp-ipc 2025-09-28 06:26:17,511 - lava_callback - ERROR - Error processing callback for node 68d633c3c1bc82a30b37ae3a: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:56532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54928 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:29:26,345 - lava_callback - INFO - Received callback for node 68d633bac1bc82a30b37ae2b INFO: 172.21.0.1:50098 - "POST /node/68d633bac1bc82a30b37ae2b HTTP/1.0" 202 Accepted 2025-09-28 06:29:26,353 - lava_callback - INFO - Starting async job submit for node 68d633bac1bc82a30b37ae2b 2025-09-28 06:29:26,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:29:26,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633bac1bc82a30b37ae2b HTTP/1.1" 200 None 2025-09-28 06:29:26,469 - lava_callback - DEBUG - Processing node 68d633bac1bc82a30b37ae2b: ltp-cve INFO: 172.21.0.1:50102 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:29:27,173 - lava_callback - ERROR - Error processing callback for node 68d633bac1bc82a30b37ae2b: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:29:45,310 - lava_callback - INFO - Received callback for node 68d8c533a178a5bbc0ea6e29 INFO: 172.21.0.1:36330 - "POST /node/68d8c533a178a5bbc0ea6e29 HTTP/1.0" 202 Accepted 2025-09-28 06:29:45,317 - lava_callback - INFO - Starting async job submit for node 68d8c533a178a5bbc0ea6e29 2025-09-28 06:29:45,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:29:45,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c533a178a5bbc0ea6e29 HTTP/1.1" 200 None 2025-09-28 06:29:45,395 - lava_callback - DEBUG - Processing node 68d8c533a178a5bbc0ea6e29: baseline-arm64 2025-09-28 06:29:45,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:29:46,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:29:46,074 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c533a178a5bbc0ea6e29/log.txt.gz 2025-09-28 06:29:46,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:29:46,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:29:46,223 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c533a178a5bbc0ea6e29/lava_callback.json.gz DEBUG: Setup failed for 68d8c533a178a5bbc0ea6e29. Transitting job node result: incomplete -> fail 2025-09-28 06:29:46,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:29:46,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c533a178a5bbc0ea6e29 HTTP/1.1" 200 None 2025-09-28 06:29:46,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:29:46,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:29:46,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:29:46,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c533a178a5bbc0ea6e29 HTTP/1.1" 200 None 2025-09-28 06:29:46,338 - lava_callback - INFO - Completed processing callback for node 68d8c533a178a5bbc0ea6e29 INFO: 172.21.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33946 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:32:35,884 - lava_callback - INFO - Received callback for node 68d63378c1bc82a30b37ad03 INFO: 172.21.0.1:51336 - "POST /node/68d63378c1bc82a30b37ad03 HTTP/1.0" 202 Accepted 2025-09-28 06:32:35,889 - lava_callback - INFO - Starting async job submit for node 68d63378c1bc82a30b37ad03 2025-09-28 06:32:35,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:35,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63378c1bc82a30b37ad03 HTTP/1.1" 200 None 2025-09-28 06:32:35,912 - lava_callback - DEBUG - Processing node 68d63378c1bc82a30b37ad03: baseline-arm 2025-09-28 06:32:35,912 - lava_callback - ERROR - Error processing callback for node 68d63378c1bc82a30b37ad03: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:51352 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:32:36,544 - lava_callback - INFO - Received callback for node 68d6337ac1bc82a30b37ad0f INFO: 172.21.0.1:51358 - "POST /node/68d6337ac1bc82a30b37ad0f HTTP/1.0" 202 Accepted 2025-09-28 06:32:36,549 - lava_callback - INFO - Starting async job submit for node 68d6337ac1bc82a30b37ad0f 2025-09-28 06:32:36,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:36,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6337ac1bc82a30b37ad0f HTTP/1.1" 200 None 2025-09-28 06:32:36,585 - lava_callback - DEBUG - Processing node 68d6337ac1bc82a30b37ad0f: baseline-arm 2025-09-28 06:32:36,585 - lava_callback - ERROR - Error processing callback for node 68d6337ac1bc82a30b37ad0f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:51360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32914 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:32:49,330 - lava_callback - INFO - Received callback for node 68d63382c1bc82a30b37ad48 INFO: 172.21.0.1:47056 - "POST /node/68d63382c1bc82a30b37ad48 HTTP/1.0" 202 Accepted 2025-09-28 06:32:49,335 - lava_callback - INFO - Starting async job submit for node 68d63382c1bc82a30b37ad48 2025-09-28 06:32:49,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:49,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63382c1bc82a30b37ad48 HTTP/1.1" 200 None 2025-09-28 06:32:49,378 - lava_callback - DEBUG - Processing node 68d63382c1bc82a30b37ad48: baseline-arm-clabbe 2025-09-28 06:32:49,378 - lava_callback - ERROR - Error processing callback for node 68d63382c1bc82a30b37ad48: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:32:49,983 - lava_callback - INFO - Received callback for node 68d63384c1bc82a30b37ad51 INFO: 172.21.0.1:47070 - "POST /node/68d63384c1bc82a30b37ad51 HTTP/1.0" 202 Accepted 2025-09-28 06:32:49,989 - lava_callback - INFO - Starting async job submit for node 68d63384c1bc82a30b37ad51 2025-09-28 06:32:49,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:50,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63384c1bc82a30b37ad51 HTTP/1.1" 200 None 2025-09-28 06:32:50,006 - lava_callback - DEBUG - Processing node 68d63384c1bc82a30b37ad51: baseline-arm-clabbe 2025-09-28 06:32:50,006 - lava_callback - ERROR - Error processing callback for node 68d63384c1bc82a30b37ad51: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:32:50,641 - lava_callback - INFO - Received callback for node 68d63385c1bc82a30b37ad5a INFO: 172.21.0.1:47078 - "POST /node/68d63385c1bc82a30b37ad5a HTTP/1.0" 202 Accepted 2025-09-28 06:32:50,646 - lava_callback - INFO - Starting async job submit for node 68d63385c1bc82a30b37ad5a 2025-09-28 06:32:50,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:50,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63385c1bc82a30b37ad5a HTTP/1.1" 200 None 2025-09-28 06:32:50,659 - lava_callback - DEBUG - Processing node 68d63385c1bc82a30b37ad5a: baseline-arm-clabbe 2025-09-28 06:32:50,659 - lava_callback - ERROR - Error processing callback for node 68d63385c1bc82a30b37ad5a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:32:51,301 - lava_callback - INFO - Received callback for node 68d63385c1bc82a30b37ad5d INFO: 172.21.0.1:47094 - "POST /node/68d63385c1bc82a30b37ad5d HTTP/1.0" 202 Accepted 2025-09-28 06:32:51,307 - lava_callback - INFO - Starting async job submit for node 68d63385c1bc82a30b37ad5d 2025-09-28 06:32:51,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:51,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63385c1bc82a30b37ad5d HTTP/1.1" 200 None 2025-09-28 06:32:51,322 - lava_callback - DEBUG - Processing node 68d63385c1bc82a30b37ad5d: baseline-arm-clabbe 2025-09-28 06:32:51,322 - lava_callback - ERROR - Error processing callback for node 68d63385c1bc82a30b37ad5d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:32:51,964 - lava_callback - INFO - Received callback for node 68d63386c1bc82a30b37ad60 INFO: 172.21.0.1:47108 - "POST /node/68d63386c1bc82a30b37ad60 HTTP/1.0" 202 Accepted 2025-09-28 06:32:51,969 - lava_callback - INFO - Starting async job submit for node 68d63386c1bc82a30b37ad60 2025-09-28 06:32:51,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:52,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63386c1bc82a30b37ad60 HTTP/1.1" 200 None 2025-09-28 06:32:52,009 - lava_callback - DEBUG - Processing node 68d63386c1bc82a30b37ad60: baseline-arm-clabbe 2025-09-28 06:32:52,009 - lava_callback - ERROR - Error processing callback for node 68d63386c1bc82a30b37ad60: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:32:52,640 - lava_callback - INFO - Received callback for node 68d63387c1bc82a30b37ad6c INFO: 172.21.0.1:47120 - "POST /node/68d63387c1bc82a30b37ad6c HTTP/1.0" 202 Accepted 2025-09-28 06:32:52,646 - lava_callback - INFO - Starting async job submit for node 68d63387c1bc82a30b37ad6c 2025-09-28 06:32:52,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:52,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63387c1bc82a30b37ad6c HTTP/1.1" 200 None 2025-09-28 06:32:52,659 - lava_callback - DEBUG - Processing node 68d63387c1bc82a30b37ad6c: baseline-arm-clabbe 2025-09-28 06:32:52,659 - lava_callback - ERROR - Error processing callback for node 68d63387c1bc82a30b37ad6c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:47136 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:32:57,276 - lava_callback - INFO - Received callback for node 68d633cec1bc82a30b37ae53 INFO: 172.21.0.1:47150 - "POST /node/68d633cec1bc82a30b37ae53 HTTP/1.0" 202 Accepted 2025-09-28 06:32:57,283 - lava_callback - INFO - Starting async job submit for node 68d633cec1bc82a30b37ae53 2025-09-28 06:32:57,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:32:57,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633cec1bc82a30b37ae53 HTTP/1.1" 200 None 2025-09-28 06:32:57,339 - lava_callback - DEBUG - Processing node 68d633cec1bc82a30b37ae53: ltp-pty 2025-09-28 06:32:57,740 - lava_callback - ERROR - Error processing callback for node 68d633cec1bc82a30b37ae53: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35094 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 06:33:16,184 - lava_callback - INFO - Received callback for node 68d63398c1bc82a30b37ade3 INFO: 172.21.0.1:35110 - "POST /node/68d63398c1bc82a30b37ade3 HTTP/1.0" 202 Accepted 2025-09-28 06:33:16,190 - lava_callback - INFO - Starting async job submit for node 68d63398c1bc82a30b37ade3 2025-09-28 06:33:16,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:33:16,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63398c1bc82a30b37ade3 HTTP/1.1" 200 None 2025-09-28 06:33:16,207 - lava_callback - DEBUG - Processing node 68d63398c1bc82a30b37ade3: baseline-arm 2025-09-28 06:33:16,207 - lava_callback - ERROR - Error processing callback for node 68d63398c1bc82a30b37ade3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:35124 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:33:16,843 - lava_callback - INFO - Received callback for node 68d63398c1bc82a30b37ade6 INFO: 172.21.0.1:35126 - "POST /node/68d63398c1bc82a30b37ade6 HTTP/1.0" 202 Accepted 2025-09-28 06:33:16,849 - lava_callback - INFO - Starting async job submit for node 68d63398c1bc82a30b37ade6 2025-09-28 06:33:16,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:33:16,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63398c1bc82a30b37ade6 HTTP/1.1" 200 None 2025-09-28 06:33:16,885 - lava_callback - DEBUG - Processing node 68d63398c1bc82a30b37ade6: baseline-arm 2025-09-28 06:33:16,885 - lava_callback - ERROR - Error processing callback for node 68d63398c1bc82a30b37ade6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:33:17,492 - lava_callback - INFO - Received callback for node 68d633a1c1bc82a30b37adfb INFO: 172.21.0.1:35140 - "POST /node/68d633a1c1bc82a30b37adfb HTTP/1.0" 202 Accepted 2025-09-28 06:33:17,498 - lava_callback - INFO - Starting async job submit for node 68d633a1c1bc82a30b37adfb 2025-09-28 06:33:17,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:33:17,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633a1c1bc82a30b37adfb HTTP/1.1" 200 None 2025-09-28 06:33:17,511 - lava_callback - DEBUG - Processing node 68d633a1c1bc82a30b37adfb: baseline-arm 2025-09-28 06:33:17,511 - lava_callback - ERROR - Error processing callback for node 68d633a1c1bc82a30b37adfb: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:33:18,146 - lava_callback - INFO - Received callback for node 68d633a1c1bc82a30b37adfe INFO: 172.21.0.1:35146 - "POST /node/68d633a1c1bc82a30b37adfe HTTP/1.0" 202 Accepted 2025-09-28 06:33:18,152 - lava_callback - INFO - Starting async job submit for node 68d633a1c1bc82a30b37adfe 2025-09-28 06:33:18,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:33:18,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633a1c1bc82a30b37adfe HTTP/1.1" 200 None 2025-09-28 06:33:18,165 - lava_callback - DEBUG - Processing node 68d633a1c1bc82a30b37adfe: baseline-arm 2025-09-28 06:33:18,165 - lava_callback - ERROR - Error processing callback for node 68d633a1c1bc82a30b37adfe: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:33:19,867 - lava_callback - INFO - Received callback for node 68d633aac1bc82a30b37ae10 INFO: 172.21.0.1:56836 - "POST /node/68d633aac1bc82a30b37ae10 HTTP/1.0" 202 Accepted 2025-09-28 06:33:19,873 - lava_callback - INFO - Starting async job submit for node 68d633aac1bc82a30b37ae10 2025-09-28 06:33:19,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:33:19,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633aac1bc82a30b37ae10 HTTP/1.1" 200 None 2025-09-28 06:33:19,891 - lava_callback - DEBUG - Processing node 68d633aac1bc82a30b37ae10: baseline-arm-broonie 2025-09-28 06:33:19,891 - lava_callback - ERROR - Error processing callback for node 68d633aac1bc82a30b37ae10: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51582 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:34:00,091 - lava_callback - INFO - Received callback for node 68d633d3c1bc82a30b37ae5c INFO: 172.21.0.1:53932 - "POST /node/68d633d3c1bc82a30b37ae5c HTTP/1.0" 202 Accepted 2025-09-28 06:34:00,097 - lava_callback - INFO - Starting async job submit for node 68d633d3c1bc82a30b37ae5c 2025-09-28 06:34:00,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:34:00,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633d3c1bc82a30b37ae5c HTTP/1.1" 200 None 2025-09-28 06:34:00,135 - lava_callback - DEBUG - Processing node 68d633d3c1bc82a30b37ae5c: ltp-syscalls 2025-09-28 06:34:00,135 - lava_callback - ERROR - Error processing callback for node 68d633d3c1bc82a30b37ae5c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:34:00,888 - lava_callback - INFO - Received callback for node 68d633d7c1bc82a30b37ae62 INFO: 172.21.0.1:53946 - "POST /node/68d633d7c1bc82a30b37ae62 HTTP/1.0" 202 Accepted 2025-09-28 06:34:00,894 - lava_callback - INFO - Starting async job submit for node 68d633d7c1bc82a30b37ae62 2025-09-28 06:34:00,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:34:00,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633d7c1bc82a30b37ae62 HTTP/1.1" 200 None 2025-09-28 06:34:00,930 - lava_callback - DEBUG - Processing node 68d633d7c1bc82a30b37ae62: ltp-timers 2025-09-28 06:34:00,930 - lava_callback - ERROR - Error processing callback for node 68d633d7c1bc82a30b37ae62: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43114 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:34:19,978 - lava_callback - INFO - Received callback for node 68d633d9c1bc82a30b37ae65 INFO: 172.21.0.1:52840 - "POST /node/68d633d9c1bc82a30b37ae65 HTTP/1.0" 202 Accepted 2025-09-28 06:34:19,984 - lava_callback - INFO - Starting async job submit for node 68d633d9c1bc82a30b37ae65 2025-09-28 06:34:19,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:34:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633d9c1bc82a30b37ae65 HTTP/1.1" 200 None 2025-09-28 06:34:19,997 - lava_callback - DEBUG - Processing node 68d633d9c1bc82a30b37ae65: ltp-watchqueue 2025-09-28 06:34:19,997 - lava_callback - ERROR - Error processing callback for node 68d633d9c1bc82a30b37ae65: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:35:30,041 - lava_callback - INFO - Received callback for node 68d8cf26a178a5bbc0ea6ed2 INFO: 172.21.0.1:40450 - "POST /node/68d8cf26a178a5bbc0ea6ed2 HTTP/1.0" 202 Accepted 2025-09-28 06:35:30,046 - lava_callback - INFO - Starting async job submit for node 68d8cf26a178a5bbc0ea6ed2 2025-09-28 06:35:30,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:35:30,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8cf26a178a5bbc0ea6ed2 HTTP/1.1" 200 None 2025-09-28 06:35:30,092 - lava_callback - DEBUG - Processing node 68d8cf26a178a5bbc0ea6ed2: baseline-arm64-broonie 2025-09-28 06:35:30,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:35:30,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:35:30,307 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8cf26a178a5bbc0ea6ed2/log.txt.gz 2025-09-28 06:35:30,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:35:30,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:35:30,447 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-68d8cf26a178a5bbc0ea6ed2/lava_callback.json.gz 2025-09-28 06:35:30,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:35:30,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8cf26a178a5bbc0ea6ed2 HTTP/1.1" 200 None 2025-09-28 06:35:30,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:35:30,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:35:30,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:35:30,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8cf26a178a5bbc0ea6ed2 HTTP/1.1" 200 None 2025-09-28 06:35:30,541 - lava_callback - INFO - Completed processing callback for node 68d8cf26a178a5bbc0ea6ed2 INFO: 172.21.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43680 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:36:36,697 - lava_callback - INFO - Received callback for node 68d816731fd44e101337f5d0 INFO: 172.21.0.1:43692 - "POST /node/68d816731fd44e101337f5d0 HTTP/1.0" 202 Accepted 2025-09-28 06:36:36,704 - lava_callback - INFO - Starting async job submit for node 68d816731fd44e101337f5d0 2025-09-28 06:36:36,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:36,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816731fd44e101337f5d0 HTTP/1.1" 200 None 2025-09-28 06:36:36,757 - lava_callback - DEBUG - Processing node 68d816731fd44e101337f5d0: baseline-arm64 2025-09-28 06:36:37,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:36:37,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:36:37,828 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816731fd44e101337f5d0/log.txt.gz 2025-09-28 06:36:37,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:36:38,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:36:38,034 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816731fd44e101337f5d0/lava_callback.json.gz DEBUG: Setup failed for 68d816731fd44e101337f5d0. Transitting job node result: incomplete -> fail 2025-09-28 06:36:38,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:38,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816731fd44e101337f5d0 HTTP/1.1" 200 None 2025-09-28 06:36:38,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:38,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:36:38,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:38,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816731fd44e101337f5d0 HTTP/1.1" 200 None 2025-09-28 06:36:38,167 - lava_callback - INFO - Completed processing callback for node 68d816731fd44e101337f5d0 INFO: 172.21.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48346 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:36:55,994 - lava_callback - INFO - Received callback for node 68d816771fd44e101337f5d3 INFO: 172.21.0.1:41722 - "POST /node/68d816771fd44e101337f5d3 HTTP/1.0" 202 Accepted 2025-09-28 06:36:56,001 - lava_callback - INFO - Starting async job submit for node 68d816771fd44e101337f5d3 2025-09-28 06:36:56,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:56,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816771fd44e101337f5d3 HTTP/1.1" 200 None 2025-09-28 06:36:56,054 - lava_callback - DEBUG - Processing node 68d816771fd44e101337f5d3: baseline-arm64 2025-09-28 06:36:56,464 - lava_callback - INFO - Received callback for node 68d63485c1bc82a30b37aeaf INFO: 172.21.0.1:41730 - "POST /node/68d63485c1bc82a30b37aeaf HTTP/1.0" 202 Accepted 2025-09-28 06:36:56,479 - lava_callback - INFO - Starting async job submit for node 68d63485c1bc82a30b37aeaf 2025-09-28 06:36:56,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41740 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:36:56,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63485c1bc82a30b37aeaf HTTP/1.1" 200 None 2025-09-28 06:36:56,575 - lava_callback - DEBUG - Processing node 68d63485c1bc82a30b37aeaf: baseline-arm 2025-09-28 06:36:56,575 - lava_callback - ERROR - Error processing callback for node 68d63485c1bc82a30b37aeaf: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:36:56,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:36:57,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:36:57,126 - lava_callback - INFO - Received callback for node 68d63485c1bc82a30b37aeb2 INFO: 172.21.0.1:41748 - "POST /node/68d63485c1bc82a30b37aeb2 HTTP/1.0" 202 Accepted 2025-09-28 06:36:57,132 - lava_callback - INFO - Starting async job submit for node 68d63485c1bc82a30b37aeb2 2025-09-28 06:36:57,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:57,133 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816771fd44e101337f5d3/log.txt.gz 2025-09-28 06:36:57,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63485c1bc82a30b37aeb2 HTTP/1.1" 200 None 2025-09-28 06:36:57,150 - lava_callback - DEBUG - Processing node 68d63485c1bc82a30b37aeb2: baseline-arm 2025-09-28 06:36:57,150 - lava_callback - ERROR - Error processing callback for node 68d63485c1bc82a30b37aeb2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:36:57,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:36:57,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:36:57,333 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816771fd44e101337f5d3/lava_callback.json.gz DEBUG: Setup failed for 68d816771fd44e101337f5d3. Transitting job node result: incomplete -> fail 2025-09-28 06:36:57,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:57,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816771fd44e101337f5d3 HTTP/1.1" 200 None 2025-09-28 06:36:57,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:57,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:36:57,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:36:57,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816771fd44e101337f5d3 HTTP/1.1" 200 None 2025-09-28 06:36:57,472 - lava_callback - INFO - Completed processing callback for node 68d816771fd44e101337f5d3 INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 06:37:16,339 - lava_callback - INFO - Received callback for node 68d6348dc1bc82a30b37aec9 INFO: 172.21.0.1:47672 - "POST /node/68d6348dc1bc82a30b37aec9 HTTP/1.0" 202 Accepted 2025-09-28 06:37:16,344 - lava_callback - INFO - Starting async job submit for node 68d6348dc1bc82a30b37aec9 2025-09-28 06:37:16,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:16,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6348dc1bc82a30b37aec9 HTTP/1.1" 200 None 2025-09-28 06:37:16,377 - lava_callback - DEBUG - Processing node 68d6348dc1bc82a30b37aec9: baseline-arm 2025-09-28 06:37:16,377 - lava_callback - ERROR - Error processing callback for node 68d6348dc1bc82a30b37aec9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:47678 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:37:17,007 - lava_callback - INFO - Received callback for node 68d6348ec1bc82a30b37aecc INFO: 172.21.0.1:47686 - "POST /node/68d6348ec1bc82a30b37aecc HTTP/1.0" 202 Accepted 2025-09-28 06:37:17,013 - lava_callback - INFO - Starting async job submit for node 68d6348ec1bc82a30b37aecc 2025-09-28 06:37:17,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:17,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6348ec1bc82a30b37aecc HTTP/1.1" 200 None 2025-09-28 06:37:17,026 - lava_callback - DEBUG - Processing node 68d6348ec1bc82a30b37aecc: baseline-arm 2025-09-28 06:37:17,026 - lava_callback - ERROR - Error processing callback for node 68d6348ec1bc82a30b37aecc: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:37:20,268 - lava_callback - INFO - Received callback for node 68d6349cc1bc82a30b37aee7 INFO: 172.21.0.1:58128 - "POST /node/68d6349cc1bc82a30b37aee7 HTTP/1.0" 202 Accepted 2025-09-28 06:37:20,274 - lava_callback - INFO - Starting async job submit for node 68d6349cc1bc82a30b37aee7 2025-09-28 06:37:20,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:20,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6349cc1bc82a30b37aee7 HTTP/1.1" 200 None 2025-09-28 06:37:20,296 - lava_callback - DEBUG - Processing node 68d6349cc1bc82a30b37aee7: baseline-arm-broonie 2025-09-28 06:37:20,296 - lava_callback - ERROR - Error processing callback for node 68d6349cc1bc82a30b37aee7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58140 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:37:29,497 - lava_callback - INFO - Received callback for node 68d634a6c1bc82a30b37aef9 INFO: 172.21.0.1:45836 - "POST /node/68d634a6c1bc82a30b37aef9 HTTP/1.0" 202 Accepted 2025-09-28 06:37:29,502 - lava_callback - INFO - Starting async job submit for node 68d634a6c1bc82a30b37aef9 2025-09-28 06:37:29,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:29,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634a6c1bc82a30b37aef9 HTTP/1.1" 200 None 2025-09-28 06:37:29,534 - lava_callback - DEBUG - Processing node 68d634a6c1bc82a30b37aef9: baseline-arm-clabbe 2025-09-28 06:37:29,534 - lava_callback - ERROR - Error processing callback for node 68d634a6c1bc82a30b37aef9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46766 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:37:40,131 - lava_callback - INFO - Received callback for node 68d634a1c1bc82a30b37aef0 INFO: 172.21.0.1:46782 - "POST /node/68d634a1c1bc82a30b37aef0 HTTP/1.0" 202 Accepted 2025-09-28 06:37:40,136 - lava_callback - INFO - Starting async job submit for node 68d634a1c1bc82a30b37aef0 2025-09-28 06:37:40,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:40,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634a1c1bc82a30b37aef0 HTTP/1.1" 200 None 2025-09-28 06:37:40,150 - lava_callback - DEBUG - Processing node 68d634a1c1bc82a30b37aef0: baseline-arm-broonie 2025-09-28 06:37:40,150 - lava_callback - ERROR - Error processing callback for node 68d634a1c1bc82a30b37aef0: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:46792 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:37:49,471 - lava_callback - INFO - Received callback for node 68d634b5c1bc82a30b37af13 INFO: 172.21.0.1:45202 - "POST /node/68d634b5c1bc82a30b37af13 HTTP/1.0" 202 Accepted 2025-09-28 06:37:49,476 - lava_callback - INFO - Starting async job submit for node 68d634b5c1bc82a30b37af13 2025-09-28 06:37:49,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:49,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634b5c1bc82a30b37af13 HTTP/1.1" 200 None 2025-09-28 06:37:49,491 - lava_callback - DEBUG - Processing node 68d634b5c1bc82a30b37af13: baseline-arm-clabbe 2025-09-28 06:37:49,491 - lava_callback - ERROR - Error processing callback for node 68d634b5c1bc82a30b37af13: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:37:50,131 - lava_callback - INFO - Received callback for node 68d634a8c1bc82a30b37aefc INFO: 172.21.0.1:45218 - "POST /node/68d634a8c1bc82a30b37aefc HTTP/1.0" 202 Accepted 2025-09-28 06:37:50,136 - lava_callback - INFO - Starting async job submit for node 68d634a8c1bc82a30b37aefc 2025-09-28 06:37:50,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:50,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634a8c1bc82a30b37aefc HTTP/1.1" 200 None 2025-09-28 06:37:50,158 - lava_callback - DEBUG - Processing node 68d634a8c1bc82a30b37aefc: baseline-arm-clabbe 2025-09-28 06:37:50,158 - lava_callback - ERROR - Error processing callback for node 68d634a8c1bc82a30b37aefc: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:37:50,783 - lava_callback - INFO - Received callback for node 68d634acc1bc82a30b37af02 INFO: 172.21.0.1:45234 - "POST /node/68d634acc1bc82a30b37af02 HTTP/1.0" 202 Accepted 2025-09-28 06:37:50,789 - lava_callback - INFO - Starting async job submit for node 68d634acc1bc82a30b37af02 2025-09-28 06:37:50,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:50,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634acc1bc82a30b37af02 HTTP/1.1" 200 None 2025-09-28 06:37:50,802 - lava_callback - DEBUG - Processing node 68d634acc1bc82a30b37af02: baseline-arm-clabbe 2025-09-28 06:37:50,802 - lava_callback - ERROR - Error processing callback for node 68d634acc1bc82a30b37af02: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:37:51,445 - lava_callback - INFO - Received callback for node 68d634adc1bc82a30b37af05 INFO: 172.21.0.1:45236 - "POST /node/68d634adc1bc82a30b37af05 HTTP/1.0" 202 Accepted 2025-09-28 06:37:51,450 - lava_callback - INFO - Starting async job submit for node 68d634adc1bc82a30b37af05 2025-09-28 06:37:51,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:51,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634adc1bc82a30b37af05 HTTP/1.1" 200 None 2025-09-28 06:37:51,473 - lava_callback - DEBUG - Processing node 68d634adc1bc82a30b37af05: baseline-arm-clabbe 2025-09-28 06:37:51,473 - lava_callback - ERROR - Error processing callback for node 68d634adc1bc82a30b37af05: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:37:52,101 - lava_callback - INFO - Received callback for node 68d634afc1bc82a30b37af08 INFO: 172.21.0.1:45242 - "POST /node/68d634afc1bc82a30b37af08 HTTP/1.0" 202 Accepted 2025-09-28 06:37:52,106 - lava_callback - INFO - Starting async job submit for node 68d634afc1bc82a30b37af08 2025-09-28 06:37:52,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:37:52,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634afc1bc82a30b37af08 HTTP/1.1" 200 None 2025-09-28 06:37:52,119 - lava_callback - DEBUG - Processing node 68d634afc1bc82a30b37af08: baseline-arm-clabbe 2025-09-28 06:37:52,119 - lava_callback - ERROR - Error processing callback for node 68d634afc1bc82a30b37af08: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:45254 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:38:00,099 - lava_callback - INFO - Received callback for node 68d634bec1bc82a30b37af1f INFO: 172.21.0.1:44406 - "POST /node/68d634bec1bc82a30b37af1f HTTP/1.0" 202 Accepted 2025-09-28 06:38:00,105 - lava_callback - INFO - Starting async job submit for node 68d634bec1bc82a30b37af1f 2025-09-28 06:38:00,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:00,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634bec1bc82a30b37af1f HTTP/1.1" 200 None 2025-09-28 06:38:00,122 - lava_callback - DEBUG - Processing node 68d634bec1bc82a30b37af1f: ltp-cap-bounds 2025-09-28 06:38:00,122 - lava_callback - ERROR - Error processing callback for node 68d634bec1bc82a30b37af1f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:00,860 - lava_callback - INFO - Received callback for node 68d634c1c1bc82a30b37af22 INFO: 172.21.0.1:44410 - "POST /node/68d634c1c1bc82a30b37af22 HTTP/1.0" 202 Accepted 2025-09-28 06:38:00,866 - lava_callback - INFO - Starting async job submit for node 68d634c1c1bc82a30b37af22 2025-09-28 06:38:00,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:00,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634c1c1bc82a30b37af22 HTTP/1.1" 200 None 2025-09-28 06:38:00,880 - lava_callback - DEBUG - Processing node 68d634c1c1bc82a30b37af22: ltp-containers 2025-09-28 06:38:00,880 - lava_callback - ERROR - Error processing callback for node 68d634c1c1bc82a30b37af22: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:01,733 - lava_callback - INFO - Received callback for node 68d634c3c1bc82a30b37af25 INFO: 172.21.0.1:44414 - "POST /node/68d634c3c1bc82a30b37af25 HTTP/1.0" 202 Accepted 2025-09-28 06:38:01,738 - lava_callback - INFO - Starting async job submit for node 68d634c3c1bc82a30b37af25 2025-09-28 06:38:01,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:01,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634c3c1bc82a30b37af25 HTTP/1.1" 200 None 2025-09-28 06:38:01,771 - lava_callback - DEBUG - Processing node 68d634c3c1bc82a30b37af25: ltp-controllers 2025-09-28 06:38:01,771 - lava_callback - ERROR - Error processing callback for node 68d634c3c1bc82a30b37af25: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:02,459 - lava_callback - INFO - Received callback for node 68d634c6c1bc82a30b37af28 INFO: 172.21.0.1:44420 - "POST /node/68d634c6c1bc82a30b37af28 HTTP/1.0" 202 Accepted 2025-09-28 06:38:02,465 - lava_callback - INFO - Starting async job submit for node 68d634c6c1bc82a30b37af28 2025-09-28 06:38:02,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:02,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634c6c1bc82a30b37af28 HTTP/1.1" 200 None 2025-09-28 06:38:02,478 - lava_callback - DEBUG - Processing node 68d634c6c1bc82a30b37af28: ltp-cve 2025-09-28 06:38:02,478 - lava_callback - ERROR - Error processing callback for node 68d634c6c1bc82a30b37af28: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:44426 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:38:06,884 - lava_callback - INFO - Received callback for node 68d633d2c1bc82a30b37ae59 INFO: 172.21.0.1:44438 - "POST /node/68d633d2c1bc82a30b37ae59 HTTP/1.0" 202 Accepted 2025-09-28 06:38:06,891 - lava_callback - INFO - Starting async job submit for node 68d633d2c1bc82a30b37ae59 2025-09-28 06:38:06,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:06,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633d2c1bc82a30b37ae59 HTTP/1.1" 200 None 2025-09-28 06:38:06,939 - lava_callback - DEBUG - Processing node 68d633d2c1bc82a30b37ae59: ltp-smoketest 2025-09-28 06:38:07,389 - lava_callback - ERROR - Error processing callback for node 68d633d2c1bc82a30b37ae59: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:38:20,257 - lava_callback - INFO - Received callback for node 68d634cdc1bc82a30b37af31 INFO: 172.21.0.1:56006 - "POST /node/68d634cdc1bc82a30b37af31 HTTP/1.0" 202 Accepted 2025-09-28 06:38:20,263 - lava_callback - INFO - Starting async job submit for node 68d634cdc1bc82a30b37af31 2025-09-28 06:38:20,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:20,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634cdc1bc82a30b37af31 HTTP/1.1" 200 None 2025-09-28 06:38:20,276 - lava_callback - DEBUG - Processing node 68d634cdc1bc82a30b37af31: ltp-fsx 2025-09-28 06:38:20,276 - lava_callback - ERROR - Error processing callback for node 68d634cdc1bc82a30b37af31: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:21,001 - lava_callback - INFO - Received callback for node 68d634cfc1bc82a30b37af34 INFO: 172.21.0.1:56010 - "POST /node/68d634cfc1bc82a30b37af34 HTTP/1.0" 202 Accepted 2025-09-28 06:38:21,007 - lava_callback - INFO - Starting async job submit for node 68d634cfc1bc82a30b37af34 2025-09-28 06:38:21,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:21,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634cfc1bc82a30b37af34 HTTP/1.1" 200 None 2025-09-28 06:38:21,020 - lava_callback - DEBUG - Processing node 68d634cfc1bc82a30b37af34: ltp-input 2025-09-28 06:38:21,020 - lava_callback - ERROR - Error processing callback for node 68d634cfc1bc82a30b37af34: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:21,735 - lava_callback - INFO - Received callback for node 68d634d2c1bc82a30b37af37 INFO: 172.21.0.1:56018 - "POST /node/68d634d2c1bc82a30b37af37 HTTP/1.0" 202 Accepted 2025-09-28 06:38:21,740 - lava_callback - INFO - Starting async job submit for node 68d634d2c1bc82a30b37af37 2025-09-28 06:38:21,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:21,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634d2c1bc82a30b37af37 HTTP/1.1" 200 None 2025-09-28 06:38:21,780 - lava_callback - DEBUG - Processing node 68d634d2c1bc82a30b37af37: ltp-ipc 2025-09-28 06:38:21,780 - lava_callback - ERROR - Error processing callback for node 68d634d2c1bc82a30b37af37: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:22,460 - lava_callback - INFO - Received callback for node 68d634d4c1bc82a30b37af3a INFO: 172.21.0.1:56030 - "POST /node/68d634d4c1bc82a30b37af3a HTTP/1.0" 202 Accepted 2025-09-28 06:38:22,466 - lava_callback - INFO - Starting async job submit for node 68d634d4c1bc82a30b37af3a 2025-09-28 06:38:22,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:22,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634d4c1bc82a30b37af3a HTTP/1.1" 200 None 2025-09-28 06:38:22,481 - lava_callback - DEBUG - Processing node 68d634d4c1bc82a30b37af3a: ltp-pty 2025-09-28 06:38:22,481 - lava_callback - ERROR - Error processing callback for node 68d634d4c1bc82a30b37af3a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:23,172 - lava_callback - INFO - Received callback for node 68d634d6c1bc82a30b37af3d INFO: 172.21.0.1:56038 - "POST /node/68d634d6c1bc82a30b37af3d HTTP/1.0" 202 Accepted 2025-09-28 06:38:23,178 - lava_callback - INFO - Starting async job submit for node 68d634d6c1bc82a30b37af3d 2025-09-28 06:38:23,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:23,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634d6c1bc82a30b37af3d HTTP/1.1" 200 None 2025-09-28 06:38:23,210 - lava_callback - DEBUG - Processing node 68d634d6c1bc82a30b37af3d: ltp-sched 2025-09-28 06:38:23,210 - lava_callback - ERROR - Error processing callback for node 68d634d6c1bc82a30b37af3d: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:23,872 - lava_callback - INFO - Received callback for node 68d634d9c1bc82a30b37af40 INFO: 172.21.0.1:56040 - "POST /node/68d634d9c1bc82a30b37af40 HTTP/1.0" 202 Accepted 2025-09-28 06:38:23,878 - lava_callback - INFO - Starting async job submit for node 68d634d9c1bc82a30b37af40 2025-09-28 06:38:23,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:23,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634d9c1bc82a30b37af40 HTTP/1.1" 200 None 2025-09-28 06:38:23,891 - lava_callback - DEBUG - Processing node 68d634d9c1bc82a30b37af40: ltp-smoketest 2025-09-28 06:38:23,891 - lava_callback - ERROR - Error processing callback for node 68d634d9c1bc82a30b37af40: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:24,617 - lava_callback - INFO - Received callback for node 68d634dbc1bc82a30b37af43 INFO: 172.21.0.1:56056 - "POST /node/68d634dbc1bc82a30b37af43 HTTP/1.0" 202 Accepted 2025-09-28 06:38:24,623 - lava_callback - INFO - Starting async job submit for node 68d634dbc1bc82a30b37af43 2025-09-28 06:38:24,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:24,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634dbc1bc82a30b37af43 HTTP/1.1" 200 None 2025-09-28 06:38:24,656 - lava_callback - DEBUG - Processing node 68d634dbc1bc82a30b37af43: ltp-syscalls 2025-09-28 06:38:24,656 - lava_callback - ERROR - Error processing callback for node 68d634dbc1bc82a30b37af43: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:25,348 - lava_callback - INFO - Received callback for node 68d634ddc1bc82a30b37af46 INFO: 172.21.0.1:56058 - "POST /node/68d634ddc1bc82a30b37af46 HTTP/1.0" 202 Accepted 2025-09-28 06:38:25,354 - lava_callback - INFO - Starting async job submit for node 68d634ddc1bc82a30b37af46 2025-09-28 06:38:25,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:25,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634ddc1bc82a30b37af46 HTTP/1.1" 200 None 2025-09-28 06:38:25,367 - lava_callback - DEBUG - Processing node 68d634ddc1bc82a30b37af46: ltp-syscalls-ipc 2025-09-28 06:38:25,367 - lava_callback - ERROR - Error processing callback for node 68d634ddc1bc82a30b37af46: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:26,060 - lava_callback - INFO - Received callback for node 68d634e0c1bc82a30b37af49 INFO: 172.21.0.1:56060 - "POST /node/68d634e0c1bc82a30b37af49 HTTP/1.0" 202 Accepted 2025-09-28 06:38:26,065 - lava_callback - INFO - Starting async job submit for node 68d634e0c1bc82a30b37af49 2025-09-28 06:38:26,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:26,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634e0c1bc82a30b37af49 HTTP/1.1" 200 None 2025-09-28 06:38:26,085 - lava_callback - DEBUG - Processing node 68d634e0c1bc82a30b37af49: ltp-timers 2025-09-28 06:38:26,085 - lava_callback - ERROR - Error processing callback for node 68d634e0c1bc82a30b37af49: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57878 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:38:36,901 - lava_callback - INFO - Received callback for node 68d8c91fa178a5bbc0ea6e67 INFO: 172.21.0.1:57894 - "POST /node/68d8c91fa178a5bbc0ea6e67 HTTP/1.0" 202 Accepted 2025-09-28 06:38:36,909 - lava_callback - INFO - Starting async job submit for node 68d8c91fa178a5bbc0ea6e67 2025-09-28 06:38:36,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:36,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c91fa178a5bbc0ea6e67 HTTP/1.1" 200 None 2025-09-28 06:38:36,961 - lava_callback - DEBUG - Processing node 68d8c91fa178a5bbc0ea6e67: baseline-arm64 2025-09-28 06:38:37,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:38:38,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:38:38,151 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c91fa178a5bbc0ea6e67/log.txt.gz 2025-09-28 06:38:38,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:38:38,309 - lava_callback - INFO - Received callback for node 68d8c91ea178a5bbc0ea6e64 INFO: 172.21.0.1:57896 - "POST /node/68d8c91ea178a5bbc0ea6e64 HTTP/1.0" 202 Accepted 2025-09-28 06:38:38,318 - lava_callback - INFO - Starting async job submit for node 68d8c91ea178a5bbc0ea6e64 2025-09-28 06:38:38,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:38,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:38:38,361 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c91fa178a5bbc0ea6e67/lava_callback.json.gz 2025-09-28 06:38:38,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c91ea178a5bbc0ea6e64 HTTP/1.1" 200 None 2025-09-28 06:38:38,441 - lava_callback - DEBUG - Processing node 68d8c91ea178a5bbc0ea6e64: baseline-arm64 DEBUG: Setup failed for 68d8c91fa178a5bbc0ea6e67. Transitting job node result: incomplete -> fail 2025-09-28 06:38:38,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:38,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c91fa178a5bbc0ea6e67 HTTP/1.1" 200 None 2025-09-28 06:38:38,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:38,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:38:38,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:38,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c91fa178a5bbc0ea6e67 HTTP/1.1" 200 None 2025-09-28 06:38:38,766 - lava_callback - INFO - Completed processing callback for node 68d8c91fa178a5bbc0ea6e67 2025-09-28 06:38:39,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:38:39,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:38:39,845 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c91ea178a5bbc0ea6e64/log.txt.gz 2025-09-28 06:38:39,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:38:40,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:38:40,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c91ea178a5bbc0ea6e64/lava_callback.json.gz DEBUG: Setup failed for 68d8c91ea178a5bbc0ea6e64. Transitting job node result: incomplete -> fail 2025-09-28 06:38:40,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:40,139 - lava_callback - INFO - Received callback for node 68d634e2c1bc82a30b37af4c INFO: 172.21.0.1:54678 - "POST /node/68d634e2c1bc82a30b37af4c HTTP/1.0" 202 Accepted 2025-09-28 06:38:40,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c91ea178a5bbc0ea6e64 HTTP/1.1" 200 None 2025-09-28 06:38:40,146 - lava_callback - INFO - Starting async job submit for node 68d634e2c1bc82a30b37af4c 2025-09-28 06:38:40,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:40,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:40,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:38:40,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:38:40,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634e2c1bc82a30b37af4c HTTP/1.1" 200 None 2025-09-28 06:38:40,163 - lava_callback - DEBUG - Processing node 68d634e2c1bc82a30b37af4c: ltp-watchqueue 2025-09-28 06:38:40,163 - lava_callback - ERROR - Error processing callback for node 68d634e2c1bc82a30b37af4c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:38:40,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c91ea178a5bbc0ea6e64 HTTP/1.1" 200 None 2025-09-28 06:38:40,187 - lava_callback - INFO - Completed processing callback for node 68d8c91ea178a5bbc0ea6e64 INFO: 172.21.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:39:07,473 - lava_callback - INFO - Received callback for node 68d8c964a178a5bbc0ea6e6d INFO: 172.21.0.1:48904 - "POST /node/68d8c964a178a5bbc0ea6e6d HTTP/1.0" 202 Accepted 2025-09-28 06:39:07,480 - lava_callback - INFO - Starting async job submit for node 68d8c964a178a5bbc0ea6e6d 2025-09-28 06:39:07,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:07,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c964a178a5bbc0ea6e6d HTTP/1.1" 200 None 2025-09-28 06:39:07,535 - lava_callback - DEBUG - Processing node 68d8c964a178a5bbc0ea6e6d: baseline-arm64 2025-09-28 06:39:08,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:39:08,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:39:08,729 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c964a178a5bbc0ea6e6d/log.txt.gz 2025-09-28 06:39:08,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:39:08,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:39:08,924 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c964a178a5bbc0ea6e6d/lava_callback.json.gz DEBUG: Setup failed for 68d8c964a178a5bbc0ea6e6d. Transitting job node result: incomplete -> fail 2025-09-28 06:39:09,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:09,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c964a178a5bbc0ea6e6d HTTP/1.1" 200 None 2025-09-28 06:39:09,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:09,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:39:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:09,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c964a178a5bbc0ea6e6d HTTP/1.1" 200 None 2025-09-28 06:39:09,077 - lava_callback - INFO - Completed processing callback for node 68d8c964a178a5bbc0ea6e6d INFO: 172.21.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:39:43,908 - lava_callback - INFO - Received callback for node 68d63522c1bc82a30b37af87 INFO: 172.21.0.1:32786 - "POST /node/68d63522c1bc82a30b37af87 HTTP/1.0" 202 Accepted 2025-09-28 06:39:43,913 - lava_callback - INFO - Starting async job submit for node 68d63522c1bc82a30b37af87 2025-09-28 06:39:43,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:43,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63522c1bc82a30b37af87 HTTP/1.1" 200 None 2025-09-28 06:39:43,927 - lava_callback - DEBUG - Processing node 68d63522c1bc82a30b37af87: baseline-arm64-baylibre 2025-09-28 06:39:43,927 - lava_callback - ERROR - Error processing callback for node 68d63522c1bc82a30b37af87: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:39:44,533 - lava_callback - INFO - Received callback for node 68d63527c1bc82a30b37afaa INFO: 172.21.0.1:32794 - "POST /node/68d63527c1bc82a30b37afaa HTTP/1.0" 202 Accepted 2025-09-28 06:39:44,538 - lava_callback - INFO - Starting async job submit for node 68d63527c1bc82a30b37afaa 2025-09-28 06:39:44,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:44,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63527c1bc82a30b37afaa HTTP/1.1" 200 None 2025-09-28 06:39:44,553 - lava_callback - DEBUG - Processing node 68d63527c1bc82a30b37afaa: baseline-arm64-baylibre 2025-09-28 06:39:44,553 - lava_callback - ERROR - Error processing callback for node 68d63527c1bc82a30b37afaa: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:32808 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:39:49,610 - lava_callback - INFO - Received callback for node 68d63531c1bc82a30b37afe3 INFO: 172.21.0.1:39996 - "POST /node/68d63531c1bc82a30b37afe3 HTTP/1.0" 202 Accepted 2025-09-28 06:39:49,616 - lava_callback - INFO - Starting async job submit for node 68d63531c1bc82a30b37afe3 2025-09-28 06:39:49,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:49,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63531c1bc82a30b37afe3 HTTP/1.1" 200 None 2025-09-28 06:39:49,630 - lava_callback - DEBUG - Processing node 68d63531c1bc82a30b37afe3: baseline-arm64-clabbe 2025-09-28 06:39:49,630 - lava_callback - ERROR - Error processing callback for node 68d63531c1bc82a30b37afe3: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:39:50,255 - lava_callback - INFO - Received callback for node 68d63531c1bc82a30b37afe6 INFO: 172.21.0.1:40006 - "POST /node/68d63531c1bc82a30b37afe6 HTTP/1.0" 202 Accepted 2025-09-28 06:39:50,261 - lava_callback - INFO - Starting async job submit for node 68d63531c1bc82a30b37afe6 2025-09-28 06:39:50,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:50,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63531c1bc82a30b37afe6 HTTP/1.1" 200 None 2025-09-28 06:39:50,274 - lava_callback - DEBUG - Processing node 68d63531c1bc82a30b37afe6: baseline-arm64-clabbe 2025-09-28 06:39:50,274 - lava_callback - ERROR - Error processing callback for node 68d63531c1bc82a30b37afe6: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:39:50,924 - lava_callback - INFO - Received callback for node 68d63532c1bc82a30b37afe9 INFO: 172.21.0.1:40020 - "POST /node/68d63532c1bc82a30b37afe9 HTTP/1.0" 202 Accepted 2025-09-28 06:39:50,929 - lava_callback - INFO - Starting async job submit for node 68d63532c1bc82a30b37afe9 2025-09-28 06:39:50,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:50,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63532c1bc82a30b37afe9 HTTP/1.1" 200 None 2025-09-28 06:39:50,946 - lava_callback - DEBUG - Processing node 68d63532c1bc82a30b37afe9: baseline-arm64-clabbe 2025-09-28 06:39:50,946 - lava_callback - ERROR - Error processing callback for node 68d63532c1bc82a30b37afe9: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:39:51,591 - lava_callback - INFO - Received callback for node 68d63533c1bc82a30b37aff2 INFO: 172.21.0.1:40022 - "POST /node/68d63533c1bc82a30b37aff2 HTTP/1.0" 202 Accepted 2025-09-28 06:39:51,597 - lava_callback - INFO - Starting async job submit for node 68d63533c1bc82a30b37aff2 2025-09-28 06:39:51,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:51,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63533c1bc82a30b37aff2 HTTP/1.1" 200 None 2025-09-28 06:39:51,613 - lava_callback - DEBUG - Processing node 68d63533c1bc82a30b37aff2: baseline-arm64-clabbe 2025-09-28 06:39:51,613 - lava_callback - ERROR - Error processing callback for node 68d63533c1bc82a30b37aff2: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:39:52,238 - lava_callback - INFO - Received callback for node 68d6352ec1bc82a30b37afda INFO: 172.21.0.1:40028 - "POST /node/68d6352ec1bc82a30b37afda HTTP/1.0" 202 Accepted 2025-09-28 06:39:52,243 - lava_callback - INFO - Starting async job submit for node 68d6352ec1bc82a30b37afda 2025-09-28 06:39:52,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:52,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6352ec1bc82a30b37afda HTTP/1.1" 200 None 2025-09-28 06:39:52,270 - lava_callback - DEBUG - Processing node 68d6352ec1bc82a30b37afda: baseline-arm64-clabbe 2025-09-28 06:39:52,270 - lava_callback - ERROR - Error processing callback for node 68d6352ec1bc82a30b37afda: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 06:39:52,880 - lava_callback - INFO - Received callback for node 68d63533c1bc82a30b37afef INFO: 172.21.0.1:40044 - "POST /node/68d63533c1bc82a30b37afef HTTP/1.0" 202 Accepted 2025-09-28 06:39:52,885 - lava_callback - INFO - Starting async job submit for node 68d63533c1bc82a30b37afef 2025-09-28 06:39:52,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:39:52,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63533c1bc82a30b37afef HTTP/1.1" 200 None 2025-09-28 06:39:52,899 - lava_callback - DEBUG - Processing node 68d63533c1bc82a30b37afef: baseline-arm64-clabbe 2025-09-28 06:39:52,899 - lava_callback - ERROR - Error processing callback for node 68d63533c1bc82a30b37afef: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58766 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:40:18,807 - lava_callback - INFO - Received callback for node 68d63394c1bc82a30b37add0 INFO: 172.21.0.1:58768 - "POST /node/68d63394c1bc82a30b37add0 HTTP/1.0" 202 Accepted 2025-09-28 06:40:18,825 - lava_callback - INFO - Starting async job submit for node 68d63394c1bc82a30b37add0 2025-09-28 06:40:20,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:40:20,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63394c1bc82a30b37add0 HTTP/1.1" 200 None 2025-09-28 06:40:20,197 - lava_callback - DEBUG - Processing node 68d63394c1bc82a30b37add0: ltp-syscalls INFO: 172.21.0.1:47126 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:40:26,860 - lava_callback - ERROR - Error processing callback for node 68d63394c1bc82a30b37add0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56420 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:41:45,398 - lava_callback - INFO - Received callback for node 68d8bcfea178a5bbc0ea6cc2 INFO: 172.21.0.1:56436 - "POST /node/68d8bcfea178a5bbc0ea6cc2 HTTP/1.0" 202 Accepted 2025-09-28 06:41:45,404 - lava_callback - INFO - Starting async job submit for node 68d8bcfea178a5bbc0ea6cc2 2025-09-28 06:41:45,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:41:45,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcfea178a5bbc0ea6cc2 HTTP/1.1" 200 None 2025-09-28 06:41:45,446 - lava_callback - DEBUG - Processing node 68d8bcfea178a5bbc0ea6cc2: baseline-arm 2025-09-28 06:41:45,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:41:45,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:41:45,885 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bcfea178a5bbc0ea6cc2/log.txt.gz 2025-09-28 06:41:45,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:41:46,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:41:46,036 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bcfea178a5bbc0ea6cc2/lava_callback.json.gz DEBUG: Setup failed for 68d8bcfea178a5bbc0ea6cc2. Transitting job node result: incomplete -> fail 2025-09-28 06:41:46,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:41:46,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcfea178a5bbc0ea6cc2 HTTP/1.1" 200 None 2025-09-28 06:41:46,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:41:46,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 06:41:46,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:41:46,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bcfea178a5bbc0ea6cc2 HTTP/1.1" 200 None 2025-09-28 06:41:46,152 - lava_callback - INFO - Completed processing callback for node 68d8bcfea178a5bbc0ea6cc2 INFO: 172.21.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49378 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:41:57,476 - lava_callback - INFO - Received callback for node 68d6402a5d233c363a762281 INFO: 172.21.0.1:49382 - "POST /node/68d6402a5d233c363a762281 HTTP/1.0" 202 Accepted 2025-09-28 06:41:57,482 - lava_callback - INFO - Starting async job submit for node 68d6402a5d233c363a762281 2025-09-28 06:41:57,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:41:57,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6402a5d233c363a762281 HTTP/1.1" 200 None 2025-09-28 06:41:57,517 - lava_callback - DEBUG - Processing node 68d6402a5d233c363a762281: baseline-arm-broonie 2025-09-28 06:41:57,746 - lava_callback - ERROR - Error processing callback for node 68d6402a5d233c363a762281: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:49308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:43:27,356 - lava_callback - INFO - Received callback for node 68d816781fd44e101337f5d6 INFO: 172.21.0.1:39556 - "POST /node/68d816781fd44e101337f5d6 HTTP/1.0" 202 Accepted 2025-09-28 06:43:27,363 - lava_callback - INFO - Starting async job submit for node 68d816781fd44e101337f5d6 2025-09-28 06:43:27,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:43:27,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816781fd44e101337f5d6 HTTP/1.1" 200 None 2025-09-28 06:43:27,417 - lava_callback - DEBUG - Processing node 68d816781fd44e101337f5d6: baseline-arm64 2025-09-28 06:43:28,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:43:28,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:43:28,422 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816781fd44e101337f5d6/log.txt.gz 2025-09-28 06:43:28,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:43:28,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:43:28,643 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816781fd44e101337f5d6/lava_callback.json.gz DEBUG: Setup failed for 68d816781fd44e101337f5d6. Transitting job node result: incomplete -> fail 2025-09-28 06:43:28,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:43:28,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816781fd44e101337f5d6 HTTP/1.1" 200 None 2025-09-28 06:43:28,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:43:28,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:43:28,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:43:28,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816781fd44e101337f5d6 HTTP/1.1" 200 None 2025-09-28 06:43:28,783 - lava_callback - INFO - Completed processing callback for node 68d816781fd44e101337f5d6 INFO: 172.21.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60506 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:45:17,786 - lava_callback - INFO - Received callback for node 68d633d0c1bc82a30b37ae56 INFO: 172.21.0.1:60520 - "POST /node/68d633d0c1bc82a30b37ae56 HTTP/1.0" 202 Accepted 2025-09-28 06:45:17,793 - lava_callback - INFO - Starting async job submit for node 68d633d0c1bc82a30b37ae56 2025-09-28 06:45:17,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:45:17,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633d0c1bc82a30b37ae56 HTTP/1.1" 200 None 2025-09-28 06:45:17,843 - lava_callback - DEBUG - Processing node 68d633d0c1bc82a30b37ae56: ltp-sched 2025-09-28 06:45:18,285 - lava_callback - ERROR - Error processing callback for node 68d633d0c1bc82a30b37ae56: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45108 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:45:31,009 - lava_callback - INFO - Received callback for node 68d634cac1bc82a30b37af2e INFO: 172.21.0.1:37276 - "POST /node/68d634cac1bc82a30b37af2e HTTP/1.0" 202 Accepted 2025-09-28 06:45:31,016 - lava_callback - INFO - Starting async job submit for node 68d634cac1bc82a30b37af2e 2025-09-28 06:45:31,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:45:31,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d634cac1bc82a30b37af2e HTTP/1.1" 200 None 2025-09-28 06:45:31,172 - lava_callback - DEBUG - Processing node 68d634cac1bc82a30b37af2e: ltp-fs-bind 2025-09-28 06:45:31,790 - lava_callback - ERROR - Error processing callback for node 68d634cac1bc82a30b37af2e: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33708 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:47:17,725 - lava_callback - INFO - Received callback for node 68d640285d233c363a762277 INFO: 172.21.0.1:33722 - "POST /node/68d640285d233c363a762277 HTTP/1.0" 202 Accepted 2025-09-28 06:47:17,731 - lava_callback - INFO - Starting async job submit for node 68d640285d233c363a762277 2025-09-28 06:47:17,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:47:17,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640285d233c363a762277 HTTP/1.1" 200 None 2025-09-28 06:47:17,766 - lava_callback - DEBUG - Processing node 68d640285d233c363a762277: baseline-arm-broonie 2025-09-28 06:47:17,972 - lava_callback - ERROR - Error processing callback for node 68d640285d233c363a762277: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42722 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:49:49,875 - lava_callback - INFO - Received callback for node 68d640365d233c363a7622c3 INFO: 172.21.0.1:53286 - "POST /node/68d640365d233c363a7622c3 HTTP/1.0" 202 Accepted 2025-09-28 06:49:49,882 - lava_callback - INFO - Starting async job submit for node 68d640365d233c363a7622c3 2025-09-28 06:49:49,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:49:49,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640365d233c363a7622c3 HTTP/1.1" 200 None 2025-09-28 06:49:49,939 - lava_callback - DEBUG - Processing node 68d640365d233c363a7622c3: ltp-cap-bounds 2025-09-28 06:49:50,273 - lava_callback - ERROR - Error processing callback for node 68d640365d233c363a7622c3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49350 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:51:58,239 - lava_callback - INFO - Received callback for node 68d633d5c1bc82a30b37ae5f INFO: 172.21.0.1:49362 - "POST /node/68d633d5c1bc82a30b37ae5f HTTP/1.0" 202 Accepted 2025-09-28 06:51:58,246 - lava_callback - INFO - Starting async job submit for node 68d633d5c1bc82a30b37ae5f 2025-09-28 06:51:58,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:51:58,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d633d5c1bc82a30b37ae5f HTTP/1.1" 200 None 2025-09-28 06:51:58,313 - lava_callback - DEBUG - Processing node 68d633d5c1bc82a30b37ae5f: ltp-syscalls-ipc 2025-09-28 06:51:59,199 - lava_callback - ERROR - Error processing callback for node 68d633d5c1bc82a30b37ae5f: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49786 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:53:26,975 - lava_callback - INFO - Received callback for node 68d8c965a178a5bbc0ea6e70 INFO: 172.21.0.1:49800 - "POST /node/68d8c965a178a5bbc0ea6e70 HTTP/1.0" 202 Accepted 2025-09-28 06:53:26,981 - lava_callback - INFO - Starting async job submit for node 68d8c965a178a5bbc0ea6e70 2025-09-28 06:53:27,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:53:27,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c965a178a5bbc0ea6e70 HTTP/1.1" 200 None 2025-09-28 06:53:27,027 - lava_callback - DEBUG - Processing node 68d8c965a178a5bbc0ea6e70: baseline-arm64 2025-09-28 06:53:27,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:53:27,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:53:27,709 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c965a178a5bbc0ea6e70/log.txt.gz 2025-09-28 06:53:27,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:53:27,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:53:27,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c965a178a5bbc0ea6e70/lava_callback.json.gz DEBUG: Setup failed for 68d8c965a178a5bbc0ea6e70. Transitting job node result: incomplete -> fail 2025-09-28 06:53:27,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:53:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c965a178a5bbc0ea6e70 HTTP/1.1" 200 None 2025-09-28 06:53:27,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:53:27,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:53:27,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:53:28,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c965a178a5bbc0ea6e70 HTTP/1.1" 200 None 2025-09-28 06:53:28,013 - lava_callback - INFO - Completed processing callback for node 68d8c965a178a5bbc0ea6e70 INFO: 172.21.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50318 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:56:18,390 - lava_callback - INFO - Received callback for node 68d640375d233c363a7622c6 INFO: 172.21.0.1:50328 - "POST /node/68d640375d233c363a7622c6 HTTP/1.0" 202 Accepted 2025-09-28 06:56:18,398 - lava_callback - INFO - Starting async job submit for node 68d640375d233c363a7622c6 2025-09-28 06:56:18,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:56:18,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640375d233c363a7622c6 HTTP/1.1" 200 None 2025-09-28 06:56:18,535 - lava_callback - DEBUG - Processing node 68d640375d233c363a7622c6: ltp-containers 2025-09-28 06:56:19,379 - lava_callback - ERROR - Error processing callback for node 68d640375d233c363a7622c6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:48046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56174 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:57:17,120 - lava_callback - INFO - Received callback for node 68d8167c1fd44e101337f5d9 INFO: 172.21.0.1:56176 - "POST /node/68d8167c1fd44e101337f5d9 HTTP/1.0" 202 Accepted 2025-09-28 06:57:17,128 - lava_callback - INFO - Starting async job submit for node 68d8167c1fd44e101337f5d9 2025-09-28 06:57:17,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:17,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8167c1fd44e101337f5d9 HTTP/1.1" 200 None 2025-09-28 06:57:17,181 - lava_callback - DEBUG - Processing node 68d8167c1fd44e101337f5d9: baseline-arm64 2025-09-28 06:57:18,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:57:18,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:57:18,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8167c1fd44e101337f5d9/log.txt.gz 2025-09-28 06:57:18,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:57:18,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:57:18,674 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8167c1fd44e101337f5d9/lava_callback.json.gz DEBUG: Setup failed for 68d8167c1fd44e101337f5d9. Transitting job node result: incomplete -> fail 2025-09-28 06:57:18,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:18,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8167c1fd44e101337f5d9 HTTP/1.1" 200 None 2025-09-28 06:57:18,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:57:18,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:18,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8167c1fd44e101337f5d9 HTTP/1.1" 200 None 2025-09-28 06:57:18,852 - lava_callback - INFO - Completed processing callback for node 68d8167c1fd44e101337f5d9 INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:57:35,941 - lava_callback - INFO - Received callback for node 68d8167d1fd44e101337f5dc INFO: 172.21.0.1:53522 - "POST /node/68d8167d1fd44e101337f5dc HTTP/1.0" 202 Accepted 2025-09-28 06:57:35,949 - lava_callback - INFO - Starting async job submit for node 68d8167d1fd44e101337f5dc 2025-09-28 06:57:35,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:36,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8167d1fd44e101337f5dc HTTP/1.1" 200 None 2025-09-28 06:57:36,005 - lava_callback - DEBUG - Processing node 68d8167d1fd44e101337f5dc: baseline-arm64 INFO: 172.21.0.1:53538 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:57:36,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:57:36,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:57:36,972 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8167d1fd44e101337f5dc/log.txt.gz 2025-09-28 06:57:36,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:57:37,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:57:37,147 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8167d1fd44e101337f5dc/lava_callback.json.gz DEBUG: Setup failed for 68d8167d1fd44e101337f5dc. Transitting job node result: incomplete -> fail 2025-09-28 06:57:37,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:37,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8167d1fd44e101337f5dc HTTP/1.1" 200 None 2025-09-28 06:57:37,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:37,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 06:57:37,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:57:37,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8167d1fd44e101337f5dc HTTP/1.1" 200 None 2025-09-28 06:57:37,284 - lava_callback - INFO - Completed processing callback for node 68d8167d1fd44e101337f5dc INFO: 172.21.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53678 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:59:17,915 - lava_callback - INFO - Received callback for node 68d8c968a178a5bbc0ea6e76 INFO: 172.21.0.1:53690 - "POST /node/68d8c968a178a5bbc0ea6e76 HTTP/1.0" 202 Accepted 2025-09-28 06:59:17,923 - lava_callback - INFO - Starting async job submit for node 68d8c968a178a5bbc0ea6e76 2025-09-28 06:59:17,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:17,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c968a178a5bbc0ea6e76 HTTP/1.1" 200 None 2025-09-28 06:59:17,978 - lava_callback - DEBUG - Processing node 68d8c968a178a5bbc0ea6e76: baseline-arm64 2025-09-28 06:59:19,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:59:19,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:59:19,199 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c968a178a5bbc0ea6e76/log.txt.gz 2025-09-28 06:59:19,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:59:19,383 - lava_callback - INFO - Received callback for node 68d8c968a178a5bbc0ea6e79 INFO: 172.21.0.1:44556 - "POST /node/68d8c968a178a5bbc0ea6e79 HTTP/1.0" 202 Accepted 2025-09-28 06:59:19,395 - lava_callback - INFO - Starting async job submit for node 68d8c968a178a5bbc0ea6e79 2025-09-28 06:59:19,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:59:19,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:19,440 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c968a178a5bbc0ea6e76/lava_callback.json.gz 2025-09-28 06:59:19,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c968a178a5bbc0ea6e79 HTTP/1.1" 200 None 2025-09-28 06:59:19,506 - lava_callback - DEBUG - Processing node 68d8c968a178a5bbc0ea6e79: baseline-arm64 DEBUG: Setup failed for 68d8c968a178a5bbc0ea6e76. Transitting job node result: incomplete -> fail 2025-09-28 06:59:19,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:19,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c968a178a5bbc0ea6e76 HTTP/1.1" 200 None 2025-09-28 06:59:19,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:19,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:59:19,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:19,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c968a178a5bbc0ea6e76 HTTP/1.1" 200 None 2025-09-28 06:59:19,848 - lava_callback - INFO - Completed processing callback for node 68d8c968a178a5bbc0ea6e76 2025-09-28 06:59:20,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:59:20,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:59:20,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c968a178a5bbc0ea6e79/log.txt.gz 2025-09-28 06:59:20,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:59:20,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:59:20,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8c968a178a5bbc0ea6e79/lava_callback.json.gz DEBUG: Setup failed for 68d8c968a178a5bbc0ea6e79. Transitting job node result: incomplete -> fail 2025-09-28 06:59:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:21,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c968a178a5bbc0ea6e79 HTTP/1.1" 200 None 2025-09-28 06:59:21,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:21,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:59:21,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:21,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c968a178a5bbc0ea6e79 HTTP/1.1" 200 None 2025-09-28 06:59:21,146 - lava_callback - INFO - Completed processing callback for node 68d8c968a178a5bbc0ea6e79 INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK 2025-09-28 06:59:28,153 - lava_callback - INFO - Received callback for node 68d8cac2a178a5bbc0ea6e82 INFO: 172.21.0.1:44582 - "POST /node/68d8cac2a178a5bbc0ea6e82 HTTP/1.0" 202 Accepted 2025-09-28 06:59:28,160 - lava_callback - INFO - Starting async job submit for node 68d8cac2a178a5bbc0ea6e82 2025-09-28 06:59:28,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:28,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8cac2a178a5bbc0ea6e82 HTTP/1.1" 200 None 2025-09-28 06:59:28,214 - lava_callback - DEBUG - Processing node 68d8cac2a178a5bbc0ea6e82: baseline-arm64 2025-09-28 06:59:29,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:59:29,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:59:29,383 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8cac2a178a5bbc0ea6e82/log.txt.gz 2025-09-28 06:59:29,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 06:59:29,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 06:59:29,559 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8cac2a178a5bbc0ea6e82/lava_callback.json.gz DEBUG: Setup failed for 68d8cac2a178a5bbc0ea6e82. Transitting job node result: incomplete -> fail 2025-09-28 06:59:29,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:29,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8cac2a178a5bbc0ea6e82 HTTP/1.1" 200 None 2025-09-28 06:59:29,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:29,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 06:59:29,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 06:59:29,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8cac2a178a5bbc0ea6e82 HTTP/1.1" 200 None 2025-09-28 06:59:29,690 - lava_callback - INFO - Completed processing callback for node 68d8cac2a178a5bbc0ea6e82 INFO: 172.21.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42916 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:01:26,766 - lava_callback - INFO - Received callback for node 68d8bcffa178a5bbc0ea6cc5 INFO: 172.21.0.1:42924 - "POST /node/68d8bcffa178a5bbc0ea6cc5 HTTP/1.0" 202 Accepted 2025-09-28 07:01:26,772 - lava_callback - INFO - Starting async job submit for node 68d8bcffa178a5bbc0ea6cc5 2025-09-28 07:01:26,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:01:26,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcffa178a5bbc0ea6cc5 HTTP/1.1" 200 None 2025-09-28 07:01:26,827 - lava_callback - DEBUG - Processing node 68d8bcffa178a5bbc0ea6cc5: baseline-arm 2025-09-28 07:01:27,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:01:27,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:01:27,240 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bcffa178a5bbc0ea6cc5/log.txt.gz 2025-09-28 07:01:27,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:01:27,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:01:27,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8bcffa178a5bbc0ea6cc5/lava_callback.json.gz DEBUG: Setup failed for 68d8bcffa178a5bbc0ea6cc5. Transitting job node result: incomplete -> fail 2025-09-28 07:01:27,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:01:27,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8bcffa178a5bbc0ea6cc5 HTTP/1.1" 200 None 2025-09-28 07:01:27,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:01:27,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 07:01:27,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:01:27,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8bcffa178a5bbc0ea6cc5 HTTP/1.1" 200 None 2025-09-28 07:01:27,531 - lava_callback - INFO - Completed processing callback for node 68d8bcffa178a5bbc0ea6cc5 INFO: 172.21.0.1:43292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:03:38,494 - lava_callback - INFO - Received callback for node 68d640395d233c363a7622d5 INFO: 172.21.0.1:39032 - "POST /node/68d640395d233c363a7622d5 HTTP/1.0" 202 Accepted 2025-09-28 07:03:38,505 - lava_callback - INFO - Starting async job submit for node 68d640395d233c363a7622d5 2025-09-28 07:03:38,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:03:38,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640395d233c363a7622d5 HTTP/1.1" 200 None 2025-09-28 07:03:38,692 - lava_callback - DEBUG - Processing node 68d640395d233c363a7622d5: ltp-fs-bind 2025-09-28 07:03:39,314 - lava_callback - ERROR - Error processing callback for node 68d640395d233c363a7622d5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:03:48,413 - lava_callback - INFO - Received callback for node 68d816a81fd44e101337f5e5 INFO: 172.21.0.1:41968 - "POST /node/68d816a81fd44e101337f5e5 HTTP/1.0" 202 Accepted 2025-09-28 07:03:48,421 - lava_callback - INFO - Starting async job submit for node 68d816a81fd44e101337f5e5 2025-09-28 07:03:48,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:03:48,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816a81fd44e101337f5e5 HTTP/1.1" 200 None 2025-09-28 07:03:48,477 - lava_callback - DEBUG - Processing node 68d816a81fd44e101337f5e5: baseline-arm64 2025-09-28 07:03:49,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:03:49,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:03:49,465 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816a81fd44e101337f5e5/log.txt.gz 2025-09-28 07:03:49,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:03:49,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:03:49,642 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816a81fd44e101337f5e5/lava_callback.json.gz DEBUG: Setup failed for 68d816a81fd44e101337f5e5. Transitting job node result: incomplete -> fail 2025-09-28 07:03:49,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:03:49,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816a81fd44e101337f5e5 HTTP/1.1" 200 None 2025-09-28 07:03:49,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:03:49,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:03:49,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:03:49,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816a81fd44e101337f5e5 HTTP/1.1" 200 None 2025-09-28 07:03:49,785 - lava_callback - INFO - Completed processing callback for node 68d816a81fd44e101337f5e5 INFO: 172.21.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59078 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:06:05,455 - lava_callback - INFO - Received callback for node 68d63b575d233c363a761ff7 INFO: 172.21.0.1:34850 - "POST /node/68d63b575d233c363a761ff7 HTTP/1.0" 202 Accepted 2025-09-28 07:06:05,460 - lava_callback - INFO - Starting async job submit for node 68d63b575d233c363a761ff7 2025-09-28 07:06:05,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:06:05,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63b575d233c363a761ff7 HTTP/1.1" 200 None 2025-09-28 07:06:05,476 - lava_callback - DEBUG - Processing node 68d63b575d233c363a761ff7: baseline-arm64-baylibre 2025-09-28 07:06:05,476 - lava_callback - ERROR - Error processing callback for node 68d63b575d233c363a761ff7: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50260 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:06:25,445 - lava_callback - INFO - Received callback for node 68d63b6e5d233c363a76200f INFO: 172.21.0.1:41404 - "POST /node/68d63b6e5d233c363a76200f HTTP/1.0" 202 Accepted 2025-09-28 07:06:25,450 - lava_callback - INFO - Starting async job submit for node 68d63b6e5d233c363a76200f 2025-09-28 07:06:25,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:06:25,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63b6e5d233c363a76200f HTTP/1.1" 200 None 2025-09-28 07:06:25,464 - lava_callback - DEBUG - Processing node 68d63b6e5d233c363a76200f: baseline-arm64-baylibre 2025-09-28 07:06:25,464 - lava_callback - ERROR - Error processing callback for node 68d63b6e5d233c363a76200f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:07:10,780 - lava_callback - INFO - Received callback for node 68d63b975d233c363a76203a INFO: 172.21.0.1:51110 - "POST /node/68d63b975d233c363a76203a HTTP/1.0" 202 Accepted 2025-09-28 07:07:10,786 - lava_callback - INFO - Starting async job submit for node 68d63b975d233c363a76203a 2025-09-28 07:07:10,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:07:10,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63b975d233c363a76203a HTTP/1.1" 200 None 2025-09-28 07:07:10,799 - lava_callback - DEBUG - Processing node 68d63b975d233c363a76203a: baseline-arm64-clabbe 2025-09-28 07:07:10,799 - lava_callback - ERROR - Error processing callback for node 68d63b975d233c363a76203a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45386 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:07:27,972 - lava_callback - INFO - Received callback for node 68d640375d233c363a7622c9 INFO: 172.21.0.1:45394 - "POST /node/68d640375d233c363a7622c9 HTTP/1.0" 202 Accepted 2025-09-28 07:07:27,983 - lava_callback - INFO - Starting async job submit for node 68d640375d233c363a7622c9 2025-09-28 07:07:28,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:07:28,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640375d233c363a7622c9 HTTP/1.1" 200 None 2025-09-28 07:07:28,378 - lava_callback - DEBUG - Processing node 68d640375d233c363a7622c9: ltp-controllers 2025-09-28 07:07:30,767 - lava_callback - INFO - Received callback for node 68d63ba25d233c363a762043 INFO: 172.21.0.1:58384 - "POST /node/68d63ba25d233c363a762043 HTTP/1.0" 202 Accepted 2025-09-28 07:07:30,782 - lava_callback - INFO - Starting async job submit for node 68d63ba25d233c363a762043 2025-09-28 07:07:30,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:07:30,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63ba25d233c363a762043 HTTP/1.1" 200 None 2025-09-28 07:07:30,888 - lava_callback - DEBUG - Processing node 68d63ba25d233c363a762043: baseline-arm64-clabbe 2025-09-28 07:07:30,888 - lava_callback - ERROR - Error processing callback for node 68d63ba25d233c363a762043: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:07:31,127 - lava_callback - ERROR - Error processing callback for node 68d640375d233c363a7622c9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-28 07:07:31,443 - lava_callback - INFO - Received callback for node 68d63b9f5d233c363a762040 INFO: 172.21.0.1:58390 - "POST /node/68d63b9f5d233c363a762040 HTTP/1.0" 202 Accepted 2025-09-28 07:07:31,448 - lava_callback - INFO - Starting async job submit for node 68d63b9f5d233c363a762040 2025-09-28 07:07:31,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:07:31,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63b9f5d233c363a762040 HTTP/1.1" 200 None 2025-09-28 07:07:31,462 - lava_callback - DEBUG - Processing node 68d63b9f5d233c363a762040: baseline-arm64-clabbe 2025-09-28 07:07:31,463 - lava_callback - ERROR - Error processing callback for node 68d63b9f5d233c363a762040: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:07:32,114 - lava_callback - INFO - Received callback for node 68d63ba65d233c363a762046 INFO: 172.21.0.1:58398 - "POST /node/68d63ba65d233c363a762046 HTTP/1.0" 202 Accepted 2025-09-28 07:07:32,119 - lava_callback - INFO - Starting async job submit for node 68d63ba65d233c363a762046 2025-09-28 07:07:32,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:07:32,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63ba65d233c363a762046 HTTP/1.1" 200 None 2025-09-28 07:07:32,134 - lava_callback - DEBUG - Processing node 68d63ba65d233c363a762046: baseline-arm64-clabbe 2025-09-28 07:07:32,134 - lava_callback - ERROR - Error processing callback for node 68d63ba65d233c363a762046: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:07:32,757 - lava_callback - INFO - Received callback for node 68d63baa5d233c363a762049 INFO: 172.21.0.1:58400 - "POST /node/68d63baa5d233c363a762049 HTTP/1.0" 202 Accepted 2025-09-28 07:07:32,762 - lava_callback - INFO - Starting async job submit for node 68d63baa5d233c363a762049 2025-09-28 07:07:32,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:07:32,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63baa5d233c363a762049 HTTP/1.1" 200 None 2025-09-28 07:07:32,786 - lava_callback - DEBUG - Processing node 68d63baa5d233c363a762049: baseline-arm64-clabbe 2025-09-28 07:07:32,786 - lava_callback - ERROR - Error processing callback for node 68d63baa5d233c363a762049: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:07:33,405 - lava_callback - INFO - Received callback for node 68d63baf5d233c363a76204c INFO: 172.21.0.1:58414 - "POST /node/68d63baf5d233c363a76204c HTTP/1.0" 202 Accepted 2025-09-28 07:07:33,410 - lava_callback - INFO - Starting async job submit for node 68d63baf5d233c363a76204c 2025-09-28 07:07:33,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:07:33,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63baf5d233c363a76204c HTTP/1.1" 200 None 2025-09-28 07:07:33,424 - lava_callback - DEBUG - Processing node 68d63baf5d233c363a76204c: baseline-arm64-clabbe 2025-09-28 07:07:33,424 - lava_callback - ERROR - Error processing callback for node 68d63baf5d233c363a76204c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45664 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:08:18,641 - lava_callback - INFO - Received callback for node 68d640395d233c363a7622d8 INFO: 172.21.0.1:45668 - "POST /node/68d640395d233c363a7622d8 HTTP/1.0" 202 Accepted 2025-09-28 07:08:18,647 - lava_callback - INFO - Starting async job submit for node 68d640395d233c363a7622d8 2025-09-28 07:08:18,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:08:18,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640395d233c363a7622d8 HTTP/1.1" 200 None 2025-09-28 07:08:18,695 - lava_callback - DEBUG - Processing node 68d640395d233c363a7622d8: ltp-fsx 2025-09-28 07:08:19,049 - lava_callback - ERROR - Error processing callback for node 68d640395d233c363a7622d8: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39544 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:11:47,940 - lava_callback - INFO - Received callback for node 68d6403a5d233c363a7622db INFO: 172.21.0.1:39558 - "POST /node/68d6403a5d233c363a7622db HTTP/1.0" 202 Accepted 2025-09-28 07:11:47,946 - lava_callback - INFO - Starting async job submit for node 68d6403a5d233c363a7622db 2025-09-28 07:11:47,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:11:48,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403a5d233c363a7622db HTTP/1.1" 200 None 2025-09-28 07:11:48,008 - lava_callback - DEBUG - Processing node 68d6403a5d233c363a7622db: ltp-input 2025-09-28 07:11:48,382 - lava_callback - ERROR - Error processing callback for node 68d6403a5d233c363a7622db: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35926 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:13:18,906 - lava_callback - INFO - Received callback for node 68d6403a5d233c363a7622de INFO: 172.21.0.1:35934 - "POST /node/68d6403a5d233c363a7622de HTTP/1.0" 202 Accepted 2025-09-28 07:13:18,913 - lava_callback - INFO - Starting async job submit for node 68d6403a5d233c363a7622de 2025-09-28 07:13:18,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:13:18,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403a5d233c363a7622de HTTP/1.1" 200 None 2025-09-28 07:13:18,951 - lava_callback - DEBUG - Processing node 68d6403a5d233c363a7622de: ltp-ipc 2025-09-28 07:13:19,302 - lava_callback - ERROR - Error processing callback for node 68d6403a5d233c363a7622de: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:13:59,140 - lava_callback - INFO - Received callback for node 68d640385d233c363a7622ce INFO: 172.21.0.1:55382 - "POST /node/68d640385d233c363a7622ce HTTP/1.0" 202 Accepted 2025-09-28 07:13:59,147 - lava_callback - INFO - Starting async job submit for node 68d640385d233c363a7622ce 2025-09-28 07:13:59,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:13:59,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640385d233c363a7622ce HTTP/1.1" 200 None 2025-09-28 07:13:59,263 - lava_callback - DEBUG - Processing node 68d640385d233c363a7622ce: ltp-cve 2025-09-28 07:13:59,981 - lava_callback - ERROR - Error processing callback for node 68d640385d233c363a7622ce: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53220 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:17:08,579 - lava_callback - INFO - Received callback for node 68d8cac3a178a5bbc0ea6e85 INFO: 172.21.0.1:53228 - "POST /node/68d8cac3a178a5bbc0ea6e85 HTTP/1.0" 202 Accepted 2025-09-28 07:17:08,585 - lava_callback - INFO - Starting async job submit for node 68d8cac3a178a5bbc0ea6e85 2025-09-28 07:17:08,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:08,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8cac3a178a5bbc0ea6e85 HTTP/1.1" 200 None 2025-09-28 07:17:08,630 - lava_callback - DEBUG - Processing node 68d8cac3a178a5bbc0ea6e85: baseline-arm64 2025-09-28 07:17:09,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:17:09,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:17:09,364 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8cac3a178a5bbc0ea6e85/log.txt.gz 2025-09-28 07:17:09,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:17:09,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:17:09,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8cac3a178a5bbc0ea6e85/lava_callback.json.gz DEBUG: Setup failed for 68d8cac3a178a5bbc0ea6e85. Transitting job node result: incomplete -> fail 2025-09-28 07:17:09,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:09,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8cac3a178a5bbc0ea6e85 HTTP/1.1" 200 None 2025-09-28 07:17:09,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:09,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aaeda178a5bbc0ea5cbd HTTP/1.1" 200 None 2025-09-28 07:17:09,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:09,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8cac3a178a5bbc0ea6e85 HTTP/1.1" 200 None 2025-09-28 07:17:09,636 - lava_callback - INFO - Completed processing callback for node 68d8cac3a178a5bbc0ea6e85 INFO: 172.21.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37376 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:17:58,165 - lava_callback - INFO - Received callback for node 68d816a91fd44e101337f5e8 INFO: 172.21.0.1:37380 - "POST /node/68d816a91fd44e101337f5e8 HTTP/1.0" 202 Accepted 2025-09-28 07:17:58,172 - lava_callback - INFO - Starting async job submit for node 68d816a91fd44e101337f5e8 2025-09-28 07:17:58,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:58,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816a91fd44e101337f5e8 HTTP/1.1" 200 None 2025-09-28 07:17:58,227 - lava_callback - DEBUG - Processing node 68d816a91fd44e101337f5e8: baseline-arm64 2025-09-28 07:17:59,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:17:59,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:17:59,240 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816a91fd44e101337f5e8/log.txt.gz 2025-09-28 07:17:59,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:17:59,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:17:59,464 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816a91fd44e101337f5e8/lava_callback.json.gz DEBUG: Setup failed for 68d816a91fd44e101337f5e8. Transitting job node result: incomplete -> fail 2025-09-28 07:17:59,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:59,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816a91fd44e101337f5e8 HTTP/1.1" 200 None 2025-09-28 07:17:59,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:59,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:17:59,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:17:59,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816a91fd44e101337f5e8 HTTP/1.1" 200 None 2025-09-28 07:17:59,612 - lava_callback - INFO - Completed processing callback for node 68d816a91fd44e101337f5e8 INFO: 172.21.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36116 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:18:16,867 - lava_callback - INFO - Received callback for node 68d816ac1fd44e101337f5eb INFO: 172.21.0.1:36124 - "POST /node/68d816ac1fd44e101337f5eb HTTP/1.0" 202 Accepted 2025-09-28 07:18:16,874 - lava_callback - INFO - Starting async job submit for node 68d816ac1fd44e101337f5eb 2025-09-28 07:18:16,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:18:16,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816ac1fd44e101337f5eb HTTP/1.1" 200 None 2025-09-28 07:18:16,929 - lava_callback - DEBUG - Processing node 68d816ac1fd44e101337f5eb: baseline-arm64 2025-09-28 07:18:17,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:18:18,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:18:18,022 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816ac1fd44e101337f5eb/log.txt.gz 2025-09-28 07:18:18,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:18:18,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:18:18,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816ac1fd44e101337f5eb/lava_callback.json.gz DEBUG: Setup failed for 68d816ac1fd44e101337f5eb. Transitting job node result: incomplete -> fail 2025-09-28 07:18:18,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:18:18,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816ac1fd44e101337f5eb HTTP/1.1" 200 None 2025-09-28 07:18:18,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:18:18,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:18:18,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:18:18,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816ac1fd44e101337f5eb HTTP/1.1" 200 None 2025-09-28 07:18:18,351 - lava_callback - INFO - Completed processing callback for node 68d816ac1fd44e101337f5eb INFO: 172.21.0.1:47372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:20:19,119 - lava_callback - INFO - Received callback for node 68d6403b5d233c363a7622e6 INFO: 172.21.0.1:54970 - "POST /node/68d6403b5d233c363a7622e6 HTTP/1.0" 202 Accepted 2025-09-28 07:20:19,126 - lava_callback - INFO - Starting async job submit for node 68d6403b5d233c363a7622e6 2025-09-28 07:20:19,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:20:19,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403b5d233c363a7622e6 HTTP/1.1" 200 None 2025-09-28 07:20:19,171 - lava_callback - DEBUG - Processing node 68d6403b5d233c363a7622e6: ltp-sched 2025-09-28 07:20:19,669 - lava_callback - ERROR - Error processing callback for node 68d6403b5d233c363a7622e6: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:33770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32936 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:21:06,894 - lava_callback - INFO - Received callback for node 68d8c1b9a178a5bbc0ea6dcb INFO: 172.21.0.1:32942 - "POST /node/68d8c1b9a178a5bbc0ea6dcb HTTP/1.0" 202 Accepted 2025-09-28 07:21:06,900 - lava_callback - INFO - Starting async job submit for node 68d8c1b9a178a5bbc0ea6dcb 2025-09-28 07:21:06,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:21:06,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1b9a178a5bbc0ea6dcb HTTP/1.1" 200 None 2025-09-28 07:21:06,942 - lava_callback - DEBUG - Processing node 68d8c1b9a178a5bbc0ea6dcb: baseline-arm 2025-09-28 07:21:07,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:21:07,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:21:07,351 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8c1b9a178a5bbc0ea6dcb/log.txt.gz 2025-09-28 07:21:07,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:21:07,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:21:07,490 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8c1b9a178a5bbc0ea6dcb/lava_callback.json.gz DEBUG: Setup failed for 68d8c1b9a178a5bbc0ea6dcb. Transitting job node result: incomplete -> fail 2025-09-28 07:21:07,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:21:07,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1b9a178a5bbc0ea6dcb HTTP/1.1" 200 None 2025-09-28 07:21:07,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:21:07,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 07:21:07,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:21:07,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c1b9a178a5bbc0ea6dcb HTTP/1.1" 200 None 2025-09-28 07:21:07,603 - lava_callback - INFO - Completed processing callback for node 68d8c1b9a178a5bbc0ea6dcb INFO: 172.21.0.1:34478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49026 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:21:28,231 - lava_callback - INFO - Received callback for node 68d6403b5d233c363a7622e1 INFO: 172.21.0.1:49036 - "POST /node/68d6403b5d233c363a7622e1 HTTP/1.0" 202 Accepted 2025-09-28 07:21:28,237 - lava_callback - INFO - Starting async job submit for node 68d6403b5d233c363a7622e1 2025-09-28 07:21:28,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:21:28,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403b5d233c363a7622e1 HTTP/1.1" 200 None 2025-09-28 07:21:28,291 - lava_callback - DEBUG - Processing node 68d6403b5d233c363a7622e1: ltp-pty 2025-09-28 07:21:28,694 - lava_callback - ERROR - Error processing callback for node 68d6403b5d233c363a7622e1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:21:59,242 - lava_callback - INFO - Received callback for node 68d6403c5d233c363a7622e9 INFO: 172.21.0.1:55744 - "POST /node/68d6403c5d233c363a7622e9 HTTP/1.0" 202 Accepted 2025-09-28 07:21:59,249 - lava_callback - INFO - Starting async job submit for node 68d6403c5d233c363a7622e9 2025-09-28 07:21:59,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:21:59,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403c5d233c363a7622e9 HTTP/1.1" 200 None 2025-09-28 07:21:59,300 - lava_callback - DEBUG - Processing node 68d6403c5d233c363a7622e9: ltp-smoketest 2025-09-28 07:21:59,706 - lava_callback - ERROR - Error processing callback for node 68d6403c5d233c363a7622e9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50658 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:24:09,228 - lava_callback - INFO - Received callback for node 68d816ad1fd44e101337f5ee INFO: 172.21.0.1:50674 - "POST /node/68d816ad1fd44e101337f5ee HTTP/1.0" 202 Accepted 2025-09-28 07:24:09,235 - lava_callback - INFO - Starting async job submit for node 68d816ad1fd44e101337f5ee 2025-09-28 07:24:09,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:24:09,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816ad1fd44e101337f5ee HTTP/1.1" 200 None 2025-09-28 07:24:09,309 - lava_callback - DEBUG - Processing node 68d816ad1fd44e101337f5ee: baseline-arm64 2025-09-28 07:24:10,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:24:10,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:24:10,350 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816ad1fd44e101337f5ee/log.txt.gz 2025-09-28 07:24:10,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:24:10,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:24:10,559 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d816ad1fd44e101337f5ee/lava_callback.json.gz DEBUG: Setup failed for 68d816ad1fd44e101337f5ee. Transitting job node result: incomplete -> fail 2025-09-28 07:24:10,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:24:10,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d816ad1fd44e101337f5ee HTTP/1.1" 200 None 2025-09-28 07:24:10,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:24:10,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:24:10,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:24:10,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d816ad1fd44e101337f5ee HTTP/1.1" 200 None 2025-09-28 07:24:10,716 - lava_callback - INFO - Completed processing callback for node 68d816ad1fd44e101337f5ee INFO: 172.21.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33560 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:25:48,433 - lava_callback - INFO - Received callback for node 68d6403e5d233c363a7622f4 INFO: 172.21.0.1:33572 - "POST /node/68d6403e5d233c363a7622f4 HTTP/1.0" 202 Accepted 2025-09-28 07:25:48,440 - lava_callback - INFO - Starting async job submit for node 68d6403e5d233c363a7622f4 2025-09-28 07:25:48,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:25:48,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403e5d233c363a7622f4 HTTP/1.1" 200 None 2025-09-28 07:25:48,496 - lava_callback - DEBUG - Processing node 68d6403e5d233c363a7622f4: ltp-timers 2025-09-28 07:25:48,854 - lava_callback - ERROR - Error processing callback for node 68d6403e5d233c363a7622f4: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54114 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:26:06,575 - lava_callback - INFO - Received callback for node 68d63ffe5d233c363a7621af INFO: 172.21.0.1:54118 - "POST /node/68d63ffe5d233c363a7621af HTTP/1.0" 202 Accepted 2025-09-28 07:26:06,580 - lava_callback - INFO - Starting async job submit for node 68d63ffe5d233c363a7621af 2025-09-28 07:26:06,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:06,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d63ffe5d233c363a7621af HTTP/1.1" 200 None 2025-09-28 07:26:06,611 - lava_callback - DEBUG - Processing node 68d63ffe5d233c363a7621af: baseline-arm64-baylibre 2025-09-28 07:26:06,611 - lava_callback - ERROR - Error processing callback for node 68d63ffe5d233c363a7621af: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:07,229 - lava_callback - INFO - Received callback for node 68d640065d233c363a7621d1 INFO: 172.21.0.1:54124 - "POST /node/68d640065d233c363a7621d1 HTTP/1.0" 202 Accepted 2025-09-28 07:26:07,234 - lava_callback - INFO - Starting async job submit for node 68d640065d233c363a7621d1 2025-09-28 07:26:07,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:07,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640065d233c363a7621d1 HTTP/1.1" 200 None 2025-09-28 07:26:07,247 - lava_callback - DEBUG - Processing node 68d640065d233c363a7621d1: baseline-arm64-baylibre 2025-09-28 07:26:07,247 - lava_callback - ERROR - Error processing callback for node 68d640065d233c363a7621d1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:11,584 - lava_callback - INFO - Received callback for node 68d640125d233c363a76220c INFO: 172.21.0.1:39198 - "POST /node/68d640125d233c363a76220c HTTP/1.0" 202 Accepted 2025-09-28 07:26:11,589 - lava_callback - INFO - Starting async job submit for node 68d640125d233c363a76220c 2025-09-28 07:26:11,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:11,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640125d233c363a76220c HTTP/1.1" 200 None 2025-09-28 07:26:11,604 - lava_callback - DEBUG - Processing node 68d640125d233c363a76220c: baseline-arm64-clabbe 2025-09-28 07:26:11,604 - lava_callback - ERROR - Error processing callback for node 68d640125d233c363a76220c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:39204 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:26:12,248 - lava_callback - INFO - Received callback for node 68d6400f5d233c363a762203 INFO: 172.21.0.1:39220 - "POST /node/68d6400f5d233c363a762203 HTTP/1.0" 202 Accepted 2025-09-28 07:26:12,254 - lava_callback - INFO - Starting async job submit for node 68d6400f5d233c363a762203 2025-09-28 07:26:12,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:12,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6400f5d233c363a762203 HTTP/1.1" 200 None 2025-09-28 07:26:12,267 - lava_callback - DEBUG - Processing node 68d6400f5d233c363a762203: baseline-arm64-clabbe 2025-09-28 07:26:12,267 - lava_callback - ERROR - Error processing callback for node 68d6400f5d233c363a762203: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:39228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39230 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:26:19,432 - lava_callback - INFO - Received callback for node 68d6403e5d233c363a7622f7 INFO: 172.21.0.1:55514 - "POST /node/68d6403e5d233c363a7622f7 HTTP/1.0" 202 Accepted 2025-09-28 07:26:19,439 - lava_callback - INFO - Starting async job submit for node 68d6403e5d233c363a7622f7 2025-09-28 07:26:19,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:19,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403e5d233c363a7622f7 HTTP/1.1" 200 None 2025-09-28 07:26:19,496 - lava_callback - DEBUG - Processing node 68d6403e5d233c363a7622f7: ltp-watchqueue 2025-09-28 07:26:19,878 - lava_callback - ERROR - Error processing callback for node 68d6403e5d233c363a7622f7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55518 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:26:31,558 - lava_callback - INFO - Received callback for node 68d640125d233c363a76220f INFO: 172.21.0.1:48318 - "POST /node/68d640125d233c363a76220f HTTP/1.0" 202 Accepted 2025-09-28 07:26:31,563 - lava_callback - INFO - Starting async job submit for node 68d640125d233c363a76220f 2025-09-28 07:26:31,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:31,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640125d233c363a76220f HTTP/1.1" 200 None 2025-09-28 07:26:31,576 - lava_callback - DEBUG - Processing node 68d640125d233c363a76220f: baseline-arm64-clabbe 2025-09-28 07:26:31,577 - lava_callback - ERROR - Error processing callback for node 68d640125d233c363a76220f: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:32,212 - lava_callback - INFO - Received callback for node 68d640145d233c363a76221b INFO: 172.21.0.1:48332 - "POST /node/68d640145d233c363a76221b HTTP/1.0" 202 Accepted 2025-09-28 07:26:32,217 - lava_callback - INFO - Starting async job submit for node 68d640145d233c363a76221b 2025-09-28 07:26:32,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:32,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640145d233c363a76221b HTTP/1.1" 200 None 2025-09-28 07:26:32,244 - lava_callback - DEBUG - Processing node 68d640145d233c363a76221b: baseline-arm64-clabbe 2025-09-28 07:26:32,244 - lava_callback - ERROR - Error processing callback for node 68d640145d233c363a76221b: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:32,893 - lava_callback - INFO - Received callback for node 68d640135d233c363a762212 INFO: 172.21.0.1:48342 - "POST /node/68d640135d233c363a762212 HTTP/1.0" 202 Accepted 2025-09-28 07:26:32,898 - lava_callback - INFO - Starting async job submit for node 68d640135d233c363a762212 2025-09-28 07:26:32,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:32,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640135d233c363a762212 HTTP/1.1" 200 None 2025-09-28 07:26:32,928 - lava_callback - DEBUG - Processing node 68d640135d233c363a762212: baseline-arm64-clabbe 2025-09-28 07:26:32,929 - lava_callback - ERROR - Error processing callback for node 68d640135d233c363a762212: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:33,564 - lava_callback - INFO - Received callback for node 68d640145d233c363a762218 INFO: 172.21.0.1:48344 - "POST /node/68d640145d233c363a762218 HTTP/1.0" 202 Accepted 2025-09-28 07:26:33,569 - lava_callback - INFO - Starting async job submit for node 68d640145d233c363a762218 2025-09-28 07:26:33,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:33,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640145d233c363a762218 HTTP/1.1" 200 None 2025-09-28 07:26:33,587 - lava_callback - DEBUG - Processing node 68d640145d233c363a762218: baseline-arm64-clabbe 2025-09-28 07:26:33,588 - lava_callback - ERROR - Error processing callback for node 68d640145d233c363a762218: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:48348 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:26:38,104 - lava_callback - INFO - Received callback for node 68d640215d233c363a762245 INFO: 172.21.0.1:48358 - "POST /node/68d640215d233c363a762245 HTTP/1.0" 202 Accepted 2025-09-28 07:26:38,109 - lava_callback - INFO - Starting async job submit for node 68d640215d233c363a762245 2025-09-28 07:26:38,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:38,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640215d233c363a762245 HTTP/1.1" 200 None 2025-09-28 07:26:38,123 - lava_callback - DEBUG - Processing node 68d640215d233c363a762245: baseline-arm 2025-09-28 07:26:38,124 - lava_callback - ERROR - Error processing callback for node 68d640215d233c363a762245: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:38,763 - lava_callback - INFO - Received callback for node 68d640235d233c363a762251 INFO: 172.21.0.1:48360 - "POST /node/68d640235d233c363a762251 HTTP/1.0" 202 Accepted 2025-09-28 07:26:38,769 - lava_callback - INFO - Starting async job submit for node 68d640235d233c363a762251 2025-09-28 07:26:38,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:38,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640235d233c363a762251 HTTP/1.1" 200 None 2025-09-28 07:26:38,782 - lava_callback - DEBUG - Processing node 68d640235d233c363a762251: baseline-arm 2025-09-28 07:26:38,782 - lava_callback - ERROR - Error processing callback for node 68d640235d233c363a762251: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:50332 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:26:51,594 - lava_callback - INFO - Received callback for node 68d6402b5d233c363a76228a INFO: 172.21.0.1:40536 - "POST /node/68d6402b5d233c363a76228a HTTP/1.0" 202 Accepted 2025-09-28 07:26:51,599 - lava_callback - INFO - Starting async job submit for node 68d6402b5d233c363a76228a 2025-09-28 07:26:51,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:51,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6402b5d233c363a76228a HTTP/1.1" 200 None 2025-09-28 07:26:51,613 - lava_callback - DEBUG - Processing node 68d6402b5d233c363a76228a: baseline-arm-clabbe 2025-09-28 07:26:51,613 - lava_callback - ERROR - Error processing callback for node 68d6402b5d233c363a76228a: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:52,263 - lava_callback - INFO - Received callback for node 68d6402d5d233c363a762293 INFO: 172.21.0.1:40546 - "POST /node/68d6402d5d233c363a762293 HTTP/1.0" 202 Accepted 2025-09-28 07:26:52,268 - lava_callback - INFO - Starting async job submit for node 68d6402d5d233c363a762293 2025-09-28 07:26:52,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:52,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6402d5d233c363a762293 HTTP/1.1" 200 None 2025-09-28 07:26:52,281 - lava_callback - DEBUG - Processing node 68d6402d5d233c363a762293: baseline-arm-clabbe 2025-09-28 07:26:52,281 - lava_callback - ERROR - Error processing callback for node 68d6402d5d233c363a762293: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40556 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:26:52,944 - lava_callback - INFO - Received callback for node 68d6402e5d233c363a762299 INFO: 172.21.0.1:40568 - "POST /node/68d6402e5d233c363a762299 HTTP/1.0" 202 Accepted 2025-09-28 07:26:52,950 - lava_callback - INFO - Starting async job submit for node 68d6402e5d233c363a762299 2025-09-28 07:26:52,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:52,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6402e5d233c363a762299 HTTP/1.1" 200 None 2025-09-28 07:26:52,963 - lava_callback - DEBUG - Processing node 68d6402e5d233c363a762299: baseline-arm-clabbe 2025-09-28 07:26:52,963 - lava_callback - ERROR - Error processing callback for node 68d6402e5d233c363a762299: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:53,611 - lava_callback - INFO - Received callback for node 68d640325d233c363a7622ab INFO: 172.21.0.1:40580 - "POST /node/68d640325d233c363a7622ab HTTP/1.0" 202 Accepted 2025-09-28 07:26:53,616 - lava_callback - INFO - Starting async job submit for node 68d640325d233c363a7622ab 2025-09-28 07:26:53,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:53,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640325d233c363a7622ab HTTP/1.1" 200 None 2025-09-28 07:26:53,630 - lava_callback - DEBUG - Processing node 68d640325d233c363a7622ab: baseline-arm-clabbe 2025-09-28 07:26:53,630 - lava_callback - ERROR - Error processing callback for node 68d640325d233c363a7622ab: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:54,274 - lava_callback - INFO - Received callback for node 68d6402f5d233c363a76229c INFO: 172.21.0.1:40592 - "POST /node/68d6402f5d233c363a76229c HTTP/1.0" 202 Accepted 2025-09-28 07:26:54,280 - lava_callback - INFO - Starting async job submit for node 68d6402f5d233c363a76229c 2025-09-28 07:26:54,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:54,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6402f5d233c363a76229c HTTP/1.1" 200 None 2025-09-28 07:26:54,293 - lava_callback - DEBUG - Processing node 68d6402f5d233c363a76229c: baseline-arm-clabbe 2025-09-28 07:26:54,293 - lava_callback - ERROR - Error processing callback for node 68d6402f5d233c363a76229c: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' 2025-09-28 07:26:54,932 - lava_callback - INFO - Received callback for node 68d640305d233c363a7622a1 INFO: 172.21.0.1:40604 - "POST /node/68d640305d233c363a7622a1 HTTP/1.0" 202 Accepted 2025-09-28 07:26:54,938 - lava_callback - INFO - Starting async job submit for node 68d640305d233c363a7622a1 2025-09-28 07:26:54,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:26:54,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d640305d233c363a7622a1 HTTP/1.1" 200 None 2025-09-28 07:26:54,951 - lava_callback - DEBUG - Processing node 68d640305d233c363a7622a1: baseline-arm-clabbe 2025-09-28 07:26:54,951 - lava_callback - ERROR - Error processing callback for node 68d640305d233c363a7622a1: 'log' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 276, in async_job_submit log_parser = job_callback.get_log_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 283, in get_log_parser return LogParser(self._data['log']) ~~~~~~~~~~^^^^^^^ KeyError: 'log' INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46186 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:27:48,033 - lava_callback - INFO - Received callback for node 68d64cb5a5c3ab168ecc79b3 INFO: 172.21.0.1:46192 - "POST /node/68d64cb5a5c3ab168ecc79b3 HTTP/1.0" 202 Accepted 2025-09-28 07:27:48,039 - lava_callback - INFO - Starting async job submit for node 68d64cb5a5c3ab168ecc79b3 2025-09-28 07:27:48,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:27:48,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cb5a5c3ab168ecc79b3 HTTP/1.1" 200 None 2025-09-28 07:27:48,073 - lava_callback - DEBUG - Processing node 68d64cb5a5c3ab168ecc79b3: baseline-arm-broonie 2025-09-28 07:27:48,264 - lava_callback - ERROR - Error processing callback for node 68d64cb5a5c3ab168ecc79b3: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35306 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:30:40,533 - lava_callback - INFO - Received callback for node 68d64cbda5c3ab168ecc79d9 INFO: 172.21.0.1:53410 - "POST /node/68d64cbda5c3ab168ecc79d9 HTTP/1.0" 202 Accepted 2025-09-28 07:30:40,539 - lava_callback - INFO - Starting async job submit for node 68d64cbda5c3ab168ecc79d9 2025-09-28 07:30:40,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:30:40,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cbda5c3ab168ecc79d9 HTTP/1.1" 200 None 2025-09-28 07:30:40,597 - lava_callback - DEBUG - Processing node 68d64cbda5c3ab168ecc79d9: ltp-cap-bounds 2025-09-28 07:30:40,922 - lava_callback - ERROR - Error processing callback for node 68d64cbda5c3ab168ecc79d9: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:53424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43734 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:32:11,444 - lava_callback - INFO - Received callback for node 68d8a4687aff7facd9a8d577 INFO: 172.21.0.1:35444 - "POST /node/68d8a4687aff7facd9a8d577 HTTP/1.0" 202 Accepted 2025-09-28 07:32:11,449 - lava_callback - INFO - Starting async job submit for node 68d8a4687aff7facd9a8d577 2025-09-28 07:32:11,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:32:11,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a4687aff7facd9a8d577 HTTP/1.1" 200 None 2025-09-28 07:32:11,504 - lava_callback - DEBUG - Processing node 68d8a4687aff7facd9a8d577: baseline-x86 2025-09-28 07:32:11,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:32:11,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:32:11,924 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8a4687aff7facd9a8d577/log.txt.gz 2025-09-28 07:32:11,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:32:12,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:32:12,069 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8a4687aff7facd9a8d577/lava_callback.json.gz 2025-09-28 07:32:12,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:32:12,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a4687aff7facd9a8d577 HTTP/1.1" 200 None 2025-09-28 07:32:12,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:32:12,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f5b7aff7facd9a8d14f HTTP/1.1" 200 None 2025-09-28 07:32:12,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:32:12,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a4687aff7facd9a8d577 HTTP/1.1" 200 None 2025-09-28 07:32:12,157 - lava_callback - INFO - Completed processing callback for node 68d8a4687aff7facd9a8d577 INFO: 172.21.0.1:35458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:33:48,848 - lava_callback - INFO - Received callback for node 68d64cbea5c3ab168ecc79dd INFO: 172.21.0.1:46322 - "POST /node/68d64cbea5c3ab168ecc79dd HTTP/1.0" 202 Accepted 2025-09-28 07:33:48,855 - lava_callback - INFO - Starting async job submit for node 68d64cbea5c3ab168ecc79dd 2025-09-28 07:33:48,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:33:48,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cbea5c3ab168ecc79dd HTTP/1.1" 200 None 2025-09-28 07:33:48,986 - lava_callback - DEBUG - Processing node 68d64cbea5c3ab168ecc79dd: ltp-containers 2025-09-28 07:33:49,776 - lava_callback - ERROR - Error processing callback for node 68d64cbea5c3ab168ecc79dd: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None 2025-09-28 07:33:53,588 - lava_callback - INFO - Received callback for node 68d8b188a178a5bbc0ea6329 INFO: 172.21.0.1:52982 - "POST /node/68d8b188a178a5bbc0ea6329 HTTP/1.0" 202 Accepted 2025-09-28 07:33:53,593 - lava_callback - INFO - Starting async job submit for node 68d8b188a178a5bbc0ea6329 2025-09-28 07:33:53,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:33:53,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b188a178a5bbc0ea6329 HTTP/1.1" 200 None 2025-09-28 07:33:53,622 - lava_callback - DEBUG - Processing node 68d8b188a178a5bbc0ea6329: baseline-x86 2025-09-28 07:33:53,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:33:54,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:33:54,063 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b188a178a5bbc0ea6329/log.txt.gz 2025-09-28 07:33:54,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:33:54,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:33:54,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-68d8b188a178a5bbc0ea6329/lava_callback.json.gz 2025-09-28 07:33:54,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:33:54,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8b188a178a5bbc0ea6329 HTTP/1.1" 200 None 2025-09-28 07:33:54,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:33:54,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8aafca178a5bbc0ea5cd2 HTTP/1.1" 200 None 2025-09-28 07:33:54,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:33:54,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8b188a178a5bbc0ea6329 HTTP/1.1" 200 None 2025-09-28 07:33:54,283 - lava_callback - INFO - Completed processing callback for node 68d8b188a178a5bbc0ea6329 INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33286 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:34:40,784 - lava_callback - INFO - Received callback for node 68d6403d5d233c363a7622ed INFO: 172.21.0.1:39454 - "POST /node/68d6403d5d233c363a7622ed HTTP/1.0" 202 Accepted 2025-09-28 07:34:40,801 - lava_callback - INFO - Starting async job submit for node 68d6403d5d233c363a7622ed 2025-09-28 07:34:42,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:34:42,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403d5d233c363a7622ed HTTP/1.1" 200 None 2025-09-28 07:34:42,176 - lava_callback - DEBUG - Processing node 68d6403d5d233c363a7622ed: ltp-syscalls INFO: 172.21.0.1:39456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39464 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:34:48,861 - lava_callback - ERROR - Error processing callback for node 68d6403d5d233c363a7622ed: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:39468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33228 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:33240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41728 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:41736 - "GET /wiki HTTP/1.0" 404 Not Found INFO: 172.21.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:36:19,599 - lava_callback - INFO - Received callback for node 68d64cb7a5c3ab168ecc79bc INFO: 172.21.0.1:54304 - "POST /node/68d64cb7a5c3ab168ecc79bc HTTP/1.0" 202 Accepted 2025-09-28 07:36:19,605 - lava_callback - INFO - Starting async job submit for node 68d64cb7a5c3ab168ecc79bc 2025-09-28 07:36:19,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:36:19,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cb7a5c3ab168ecc79bc HTTP/1.1" 200 None 2025-09-28 07:36:19,637 - lava_callback - DEBUG - Processing node 68d64cb7a5c3ab168ecc79bc: baseline-arm-broonie 2025-09-28 07:36:19,852 - lava_callback - ERROR - Error processing callback for node 68d64cb7a5c3ab168ecc79bc: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50868 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:38:38,928 - lava_callback - INFO - Received callback for node 68d81b1e1fd44e101337f792 INFO: 172.21.0.1:50876 - "POST /node/68d81b1e1fd44e101337f792 HTTP/1.0" 202 Accepted 2025-09-28 07:38:38,934 - lava_callback - INFO - Starting async job submit for node 68d81b1e1fd44e101337f792 2025-09-28 07:38:38,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:38,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b1e1fd44e101337f792 HTTP/1.1" 200 None 2025-09-28 07:38:38,988 - lava_callback - DEBUG - Processing node 68d81b1e1fd44e101337f792: baseline-arm64 2025-09-28 07:38:39,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:38:40,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:38:40,000 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b1e1fd44e101337f792/log.txt.gz 2025-09-28 07:38:40,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:38:40,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:38:40,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b1e1fd44e101337f792/lava_callback.json.gz DEBUG: Setup failed for 68d81b1e1fd44e101337f792. Transitting job node result: incomplete -> fail 2025-09-28 07:38:40,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:40,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b1e1fd44e101337f792 HTTP/1.1" 200 None 2025-09-28 07:38:40,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:40,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:38:40,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:40,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d81b1e1fd44e101337f792 HTTP/1.1" 200 None 2025-09-28 07:38:40,305 - lava_callback - INFO - Completed processing callback for node 68d81b1e1fd44e101337f792 INFO: 172.21.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:38:58,080 - lava_callback - INFO - Received callback for node 68d81b1f1fd44e101337f795 INFO: 172.21.0.1:55308 - "POST /node/68d81b1f1fd44e101337f795 HTTP/1.0" 202 Accepted 2025-09-28 07:38:58,087 - lava_callback - INFO - Starting async job submit for node 68d81b1f1fd44e101337f795 2025-09-28 07:38:58,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:58,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b1f1fd44e101337f795 HTTP/1.1" 200 None 2025-09-28 07:38:58,159 - lava_callback - DEBUG - Processing node 68d81b1f1fd44e101337f795: baseline-arm64 2025-09-28 07:38:59,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:38:59,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:38:59,160 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b1f1fd44e101337f795/log.txt.gz 2025-09-28 07:38:59,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:38:59,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:38:59,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b1f1fd44e101337f795/lava_callback.json.gz DEBUG: Setup failed for 68d81b1f1fd44e101337f795. Transitting job node result: incomplete -> fail 2025-09-28 07:38:59,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:59,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b1f1fd44e101337f795 HTTP/1.1" 200 None 2025-09-28 07:38:59,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:59,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:38:59,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:38:59,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d81b1f1fd44e101337f795 HTTP/1.1" 200 None 2025-09-28 07:38:59,529 - lava_callback - INFO - Completed processing callback for node 68d81b1f1fd44e101337f795 INFO: 172.21.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40514 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:39:20,098 - lava_callback - INFO - Received callback for node 68d6403d5d233c363a7622f1 INFO: 172.21.0.1:38452 - "POST /node/68d6403d5d233c363a7622f1 HTTP/1.0" 202 Accepted 2025-09-28 07:39:20,106 - lava_callback - INFO - Starting async job submit for node 68d6403d5d233c363a7622f1 2025-09-28 07:39:20,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:39:20,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d6403d5d233c363a7622f1 HTTP/1.1" 200 None 2025-09-28 07:39:20,169 - lava_callback - DEBUG - Processing node 68d6403d5d233c363a7622f1: ltp-syscalls-ipc 2025-09-28 07:39:21,060 - lava_callback - ERROR - Error processing callback for node 68d6403d5d233c363a7622f1: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:38454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36080 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:40:47,630 - lava_callback - INFO - Received callback for node 68d8c1baa178a5bbc0ea6dce INFO: 172.21.0.1:36084 - "POST /node/68d8c1baa178a5bbc0ea6dce HTTP/1.0" 202 Accepted 2025-09-28 07:40:47,636 - lava_callback - INFO - Starting async job submit for node 68d8c1baa178a5bbc0ea6dce 2025-09-28 07:40:47,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:40:47,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1baa178a5bbc0ea6dce HTTP/1.1" 200 None 2025-09-28 07:40:47,684 - lava_callback - DEBUG - Processing node 68d8c1baa178a5bbc0ea6dce: baseline-arm 2025-09-28 07:40:47,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:40:48,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:40:48,131 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8c1baa178a5bbc0ea6dce/log.txt.gz 2025-09-28 07:40:48,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:40:48,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:40:48,291 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8c1baa178a5bbc0ea6dce/lava_callback.json.gz DEBUG: Setup failed for 68d8c1baa178a5bbc0ea6dce. Transitting job node result: incomplete -> fail 2025-09-28 07:40:48,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:40:48,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c1baa178a5bbc0ea6dce HTTP/1.1" 200 None 2025-09-28 07:40:48,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:40:48,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 07:40:48,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:40:48,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c1baa178a5bbc0ea6dce HTTP/1.1" 200 None 2025-09-28 07:40:48,423 - lava_callback - INFO - Completed processing callback for node 68d8c1baa178a5bbc0ea6dce INFO: 172.21.0.1:34444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44548 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:43:44,427 - lava_callback - INFO - Received callback for node 68d826e71fd44e101337f8c5 INFO: 172.21.0.1:47564 - "POST /node/68d826e71fd44e101337f8c5 HTTP/1.0" 202 Accepted 2025-09-28 07:43:44,433 - lava_callback - INFO - Starting async job submit for node 68d826e71fd44e101337f8c5 2025-09-28 07:43:44,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:43:44,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d826e71fd44e101337f8c5 HTTP/1.1" 200 None 2025-09-28 07:43:44,474 - lava_callback - DEBUG - Processing node 68d826e71fd44e101337f8c5: baseline-arm64 2025-09-28 07:43:44,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:43:44,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:43:44,959 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d826e71fd44e101337f8c5/log.txt.gz 2025-09-28 07:43:44,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:43:45,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:43:45,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d826e71fd44e101337f8c5/lava_callback.json.gz 2025-09-28 07:43:45,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:43:45,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d826e71fd44e101337f8c5 HTTP/1.1" 200 None 2025-09-28 07:43:45,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:43:45,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d80f531fd44e101337efb0 HTTP/1.1" 200 None 2025-09-28 07:43:45,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:43:45,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d826e71fd44e101337f8c5 HTTP/1.1" 200 None 2025-09-28 07:43:45,202 - lava_callback - INFO - Completed processing callback for node 68d826e71fd44e101337f8c5 INFO: 172.21.0.1:47580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45128 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:44:49,733 - lava_callback - INFO - Received callback for node 68d81b211fd44e101337f798 INFO: 172.21.0.1:38170 - "POST /node/68d81b211fd44e101337f798 HTTP/1.0" 202 Accepted 2025-09-28 07:44:49,740 - lava_callback - INFO - Starting async job submit for node 68d81b211fd44e101337f798 2025-09-28 07:44:49,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:44:49,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b211fd44e101337f798 HTTP/1.1" 200 None 2025-09-28 07:44:49,811 - lava_callback - DEBUG - Processing node 68d81b211fd44e101337f798: baseline-arm64 2025-09-28 07:44:50,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:44:50,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:44:50,855 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b211fd44e101337f798/log.txt.gz 2025-09-28 07:44:50,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:44:51,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:44:51,076 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b211fd44e101337f798/lava_callback.json.gz DEBUG: Setup failed for 68d81b211fd44e101337f798. Transitting job node result: incomplete -> fail 2025-09-28 07:44:51,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:44:51,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b211fd44e101337f798 HTTP/1.1" 200 None 2025-09-28 07:44:51,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:44:51,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:44:51,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:44:51,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d81b211fd44e101337f798 HTTP/1.1" 200 None 2025-09-28 07:44:51,211 - lava_callback - INFO - Completed processing callback for node 68d81b211fd44e101337f798 INFO: 172.21.0.1:38180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:45:04,754 - lava_callback - INFO - Received callback for node 68d826eb1fd44e101337f8e0 INFO: 172.21.0.1:60166 - "POST /node/68d826eb1fd44e101337f8e0 HTTP/1.0" 202 Accepted 2025-09-28 07:45:04,760 - lava_callback - INFO - Starting async job submit for node 68d826eb1fd44e101337f8e0 2025-09-28 07:45:04,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:45:04,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d826eb1fd44e101337f8e0 HTTP/1.1" 200 None 2025-09-28 07:45:04,814 - lava_callback - DEBUG - Processing node 68d826eb1fd44e101337f8e0: baseline-arm64 2025-09-28 07:45:05,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:45:05,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:45:05,302 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d826eb1fd44e101337f8e0/log.txt.gz 2025-09-28 07:45:05,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:45:05,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:45:05,423 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d826eb1fd44e101337f8e0/lava_callback.json.gz 2025-09-28 07:45:05,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:45:05,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d826eb1fd44e101337f8e0 HTTP/1.1" 200 None 2025-09-28 07:45:05,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:45:05,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d80f531fd44e101337efb0 HTTP/1.1" 200 None 2025-09-28 07:45:05,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:45:05,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d826eb1fd44e101337f8e0 HTTP/1.1" 200 None 2025-09-28 07:45:05,508 - lava_callback - INFO - Completed processing callback for node 68d826eb1fd44e101337f8e0 INFO: 172.21.0.1:60176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39904 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:46:04,042 - lava_callback - INFO - Received callback for node 68d8343f1fd44e101337fd50 INFO: 172.21.0.1:37880 - "POST /node/68d8343f1fd44e101337fd50 HTTP/1.0" 202 Accepted 2025-09-28 07:46:04,048 - lava_callback - INFO - Starting async job submit for node 68d8343f1fd44e101337fd50 2025-09-28 07:46:04,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:04,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8343f1fd44e101337fd50 HTTP/1.1" 200 None 2025-09-28 07:46:04,082 - lava_callback - DEBUG - Processing node 68d8343f1fd44e101337fd50: baseline-arm64 2025-09-28 07:46:04,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:46:04,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:46:04,523 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8343f1fd44e101337fd50/log.txt.gz 2025-09-28 07:46:04,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:46:04,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:46:04,651 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8343f1fd44e101337fd50/lava_callback.json.gz 2025-09-28 07:46:04,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:04,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8343f1fd44e101337fd50 HTTP/1.1" 200 None 2025-09-28 07:46:04,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:04,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:46:04,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:04,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8343f1fd44e101337fd50 HTTP/1.1" 200 None 2025-09-28 07:46:04,767 - lava_callback - INFO - Completed processing callback for node 68d8343f1fd44e101337fd50 INFO: 172.21.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56894 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:46:24,177 - lava_callback - INFO - Received callback for node 68d834401fd44e101337fd53 INFO: 172.21.0.1:56400 - "POST /node/68d834401fd44e101337fd53 HTTP/1.0" 202 Accepted 2025-09-28 07:46:24,183 - lava_callback - INFO - Starting async job submit for node 68d834401fd44e101337fd53 2025-09-28 07:46:24,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:24,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834401fd44e101337fd53 HTTP/1.1" 200 None 2025-09-28 07:46:24,223 - lava_callback - DEBUG - Processing node 68d834401fd44e101337fd53: baseline-arm64 2025-09-28 07:46:24,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:46:24,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:46:24,691 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834401fd44e101337fd53/log.txt.gz 2025-09-28 07:46:24,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:46:24,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:46:24,832 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834401fd44e101337fd53/lava_callback.json.gz 2025-09-28 07:46:24,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:24,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834401fd44e101337fd53 HTTP/1.1" 200 None 2025-09-28 07:46:24,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:24,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:46:24,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:24,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834401fd44e101337fd53 HTTP/1.1" 200 None 2025-09-28 07:46:24,906 - lava_callback - INFO - Completed processing callback for node 68d834401fd44e101337fd53 INFO: 172.21.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:46:40,188 - lava_callback - INFO - Received callback for node 68d64cc0a5c3ab168ecc79ec INFO: 172.21.0.1:44422 - "POST /node/68d64cc0a5c3ab168ecc79ec HTTP/1.0" 202 Accepted 2025-09-28 07:46:40,196 - lava_callback - INFO - Starting async job submit for node 68d64cc0a5c3ab168ecc79ec 2025-09-28 07:46:40,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:46:40,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc0a5c3ab168ecc79ec HTTP/1.1" 200 None 2025-09-28 07:46:40,321 - lava_callback - DEBUG - Processing node 68d64cc0a5c3ab168ecc79ec: ltp-fs-bind 2025-09-28 07:46:40,861 - lava_callback - ERROR - Error processing callback for node 68d64cc0a5c3ab168ecc79ec: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50580 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:47:00,974 - lava_callback - INFO - Received callback for node 68d64cbea5c3ab168ecc79e0 INFO: 172.21.0.1:57202 - "POST /node/68d64cbea5c3ab168ecc79e0 HTTP/1.0" 202 Accepted 2025-09-28 07:47:00,985 - lava_callback - INFO - Starting async job submit for node 68d64cbea5c3ab168ecc79e0 2025-09-28 07:47:01,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:01,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cbea5c3ab168ecc79e0 HTTP/1.1" 200 None 2025-09-28 07:47:01,335 - lava_callback - DEBUG - Processing node 68d64cbea5c3ab168ecc79e0: ltp-controllers 2025-09-28 07:47:04,079 - lava_callback - ERROR - Error processing callback for node 68d64cbea5c3ab168ecc79e0: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52562 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 07:47:15,558 - lava_callback - INFO - Received callback for node 68d834421fd44e101337fd56 INFO: 172.21.0.1:52564 - "POST /node/68d834421fd44e101337fd56 HTTP/1.0" 202 Accepted 2025-09-28 07:47:15,564 - lava_callback - INFO - Starting async job submit for node 68d834421fd44e101337fd56 2025-09-28 07:47:15,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:15,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834421fd44e101337fd56 HTTP/1.1" 200 None 2025-09-28 07:47:15,634 - lava_callback - DEBUG - Processing node 68d834421fd44e101337fd56: baseline-arm64 2025-09-28 07:47:15,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:47:16,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:47:16,081 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834421fd44e101337fd56/log.txt.gz 2025-09-28 07:47:16,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:47:16,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:47:16,224 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834421fd44e101337fd56/lava_callback.json.gz 2025-09-28 07:47:16,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:16,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834421fd44e101337fd56 HTTP/1.1" 200 None 2025-09-28 07:47:16,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:16,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:47:16,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:16,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834421fd44e101337fd56 HTTP/1.1" 200 None 2025-09-28 07:47:16,309 - lava_callback - INFO - Completed processing callback for node 68d834421fd44e101337fd56 INFO: 172.21.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40060 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:47:49,551 - lava_callback - INFO - Received callback for node 68d834431fd44e101337fd59 INFO: 172.21.0.1:52568 - "POST /node/68d834431fd44e101337fd59 HTTP/1.0" 202 Accepted 2025-09-28 07:47:49,557 - lava_callback - INFO - Starting async job submit for node 68d834431fd44e101337fd59 2025-09-28 07:47:49,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:49,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834431fd44e101337fd59 HTTP/1.1" 200 None 2025-09-28 07:47:49,594 - lava_callback - DEBUG - Processing node 68d834431fd44e101337fd59: baseline-arm64 2025-09-28 07:47:49,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:47:50,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:47:50,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834431fd44e101337fd59/log.txt.gz 2025-09-28 07:47:50,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:47:50,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:47:50,181 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834431fd44e101337fd59/lava_callback.json.gz 2025-09-28 07:47:50,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:50,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834431fd44e101337fd59 HTTP/1.1" 200 None 2025-09-28 07:47:50,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:50,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:47:50,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:47:50,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834431fd44e101337fd59 HTTP/1.1" 200 None 2025-09-28 07:47:50,258 - lava_callback - INFO - Completed processing callback for node 68d834431fd44e101337fd59 INFO: 172.21.0.1:52572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44902 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 07:48:15,466 - lava_callback - INFO - Received callback for node 68d834461fd44e101337fd5c INFO: 172.21.0.1:44904 - "POST /node/68d834461fd44e101337fd5c HTTP/1.0" 202 Accepted 2025-09-28 07:48:15,472 - lava_callback - INFO - Starting async job submit for node 68d834461fd44e101337fd5c 2025-09-28 07:48:15,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:48:15,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834461fd44e101337fd5c HTTP/1.1" 200 None 2025-09-28 07:48:15,505 - lava_callback - DEBUG - Processing node 68d834461fd44e101337fd5c: baseline-arm64 2025-09-28 07:48:15,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:48:15,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:48:15,957 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834461fd44e101337fd5c/log.txt.gz 2025-09-28 07:48:15,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:48:16,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:48:16,114 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834461fd44e101337fd5c/lava_callback.json.gz 2025-09-28 07:48:16,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:48:16,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834461fd44e101337fd5c HTTP/1.1" 200 None 2025-09-28 07:48:16,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:48:16,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:48:16,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:48:16,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834461fd44e101337fd5c HTTP/1.1" 200 None 2025-09-28 07:48:16,234 - lava_callback - INFO - Completed processing callback for node 68d834461fd44e101337fd5c INFO: 172.21.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59624 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:49:11,951 - lava_callback - INFO - Received callback for node 68d834461fd44e101337fd5f INFO: 172.21.0.1:52576 - "POST /node/68d834461fd44e101337fd5f HTTP/1.0" 202 Accepted 2025-09-28 07:49:11,956 - lava_callback - INFO - Starting async job submit for node 68d834461fd44e101337fd5f 2025-09-28 07:49:11,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:11,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834461fd44e101337fd5f HTTP/1.1" 200 None 2025-09-28 07:49:12,000 - lava_callback - DEBUG - Processing node 68d834461fd44e101337fd5f: baseline-arm64 2025-09-28 07:49:12,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:49:12,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:49:12,423 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834461fd44e101337fd5f/log.txt.gz 2025-09-28 07:49:12,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:52578 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:49:12,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:49:12,679 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834461fd44e101337fd5f/lava_callback.json.gz 2025-09-28 07:49:12,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:12,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834461fd44e101337fd5f HTTP/1.1" 200 None 2025-09-28 07:49:12,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:12,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:49:12,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:12,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834461fd44e101337fd5f HTTP/1.1" 200 None 2025-09-28 07:49:12,753 - lava_callback - INFO - Completed processing callback for node 68d834461fd44e101337fd5f INFO: 172.21.0.1:52580 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 07:49:15,544 - lava_callback - INFO - Received callback for node 68d834491fd44e101337fd62 INFO: 172.21.0.1:52588 - "POST /node/68d834491fd44e101337fd62 HTTP/1.0" 202 Accepted 2025-09-28 07:49:15,550 - lava_callback - INFO - Starting async job submit for node 68d834491fd44e101337fd62 2025-09-28 07:49:15,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:15,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834491fd44e101337fd62 HTTP/1.1" 200 None 2025-09-28 07:49:15,590 - lava_callback - DEBUG - Processing node 68d834491fd44e101337fd62: baseline-arm64 2025-09-28 07:49:15,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:49:16,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:49:16,031 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834491fd44e101337fd62/log.txt.gz 2025-09-28 07:49:16,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:49:16,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:49:16,151 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834491fd44e101337fd62/lava_callback.json.gz 2025-09-28 07:49:16,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:16,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834491fd44e101337fd62 HTTP/1.1" 200 None 2025-09-28 07:49:16,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:16,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:49:16,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:49:16,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834491fd44e101337fd62 HTTP/1.1" 200 None 2025-09-28 07:49:16,268 - lava_callback - INFO - Completed processing callback for node 68d834491fd44e101337fd62 INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52034 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 07:50:15,420 - lava_callback - INFO - Received callback for node 68d834491fd44e101337fd65 INFO: 172.21.0.1:52040 - "POST /node/68d834491fd44e101337fd65 HTTP/1.0" 202 Accepted 2025-09-28 07:50:15,427 - lava_callback - INFO - Starting async job submit for node 68d834491fd44e101337fd65 2025-09-28 07:50:15,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:15,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834491fd44e101337fd65 HTTP/1.1" 200 None 2025-09-28 07:50:15,462 - lava_callback - DEBUG - Processing node 68d834491fd44e101337fd65: baseline-arm64 2025-09-28 07:50:15,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:50:15,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:50:15,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834491fd44e101337fd65/log.txt.gz 2025-09-28 07:50:15,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:50:16,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:50:16,058 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834491fd44e101337fd65/lava_callback.json.gz 2025-09-28 07:50:16,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:16,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834491fd44e101337fd65 HTTP/1.1" 200 None 2025-09-28 07:50:16,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:16,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:50:16,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:16,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834491fd44e101337fd65 HTTP/1.1" 200 None 2025-09-28 07:50:16,134 - lava_callback - INFO - Completed processing callback for node 68d834491fd44e101337fd65 INFO: 172.21.0.1:52050 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:50:26,211 - lava_callback - INFO - Received callback for node 68d8347b1fd44e101337fd70 INFO: 172.21.0.1:44742 - "POST /node/68d8347b1fd44e101337fd70 HTTP/1.0" 202 Accepted 2025-09-28 07:50:26,217 - lava_callback - INFO - Starting async job submit for node 68d8347b1fd44e101337fd70 2025-09-28 07:50:26,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:26,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347b1fd44e101337fd70 HTTP/1.1" 200 None 2025-09-28 07:50:26,286 - lava_callback - DEBUG - Processing node 68d8347b1fd44e101337fd70: baseline-arm64 INFO: 172.21.0.1:44752 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:50:26,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:50:26,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:50:26,718 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347b1fd44e101337fd70/log.txt.gz 2025-09-28 07:50:26,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:50:26,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:50:26,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347b1fd44e101337fd70/lava_callback.json.gz 2025-09-28 07:50:26,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:26,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347b1fd44e101337fd70 HTTP/1.1" 200 None 2025-09-28 07:50:26,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:26,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:50:26,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:50:26,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8347b1fd44e101337fd70 HTTP/1.1" 200 None 2025-09-28 07:50:26,923 - lava_callback - INFO - Completed processing callback for node 68d8347b1fd44e101337fd70 INFO: 172.21.0.1:38786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47096 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:51:00,317 - lava_callback - INFO - Received callback for node 68d64cc0a5c3ab168ecc79ef INFO: 172.21.0.1:55758 - "POST /node/68d64cc0a5c3ab168ecc79ef HTTP/1.0" 202 Accepted 2025-09-28 07:51:00,323 - lava_callback - INFO - Starting async job submit for node 68d64cc0a5c3ab168ecc79ef 2025-09-28 07:51:00,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:00,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc0a5c3ab168ecc79ef HTTP/1.1" 200 None 2025-09-28 07:51:00,380 - lava_callback - DEBUG - Processing node 68d64cc0a5c3ab168ecc79ef: ltp-fsx 2025-09-28 07:51:00,731 - lava_callback - ERROR - Error processing callback for node 68d64cc0a5c3ab168ecc79ef: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 07:51:15,538 - lava_callback - INFO - Received callback for node 68d8347c1fd44e101337fd75 INFO: 172.21.0.1:33822 - "POST /node/68d8347c1fd44e101337fd75 HTTP/1.0" 202 Accepted 2025-09-28 07:51:15,544 - lava_callback - INFO - Starting async job submit for node 68d8347c1fd44e101337fd75 2025-09-28 07:51:15,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:15,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347c1fd44e101337fd75 HTTP/1.1" 200 None 2025-09-28 07:51:15,578 - lava_callback - DEBUG - Processing node 68d8347c1fd44e101337fd75: baseline-arm64 2025-09-28 07:51:15,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:51:16,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:51:16,025 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347c1fd44e101337fd75/log.txt.gz 2025-09-28 07:51:16,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:51:16,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:51:16,144 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347c1fd44e101337fd75/lava_callback.json.gz 2025-09-28 07:51:16,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:16,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347c1fd44e101337fd75 HTTP/1.1" 200 None 2025-09-28 07:51:16,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:16,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:51:16,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:16,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8347c1fd44e101337fd75 HTTP/1.1" 200 None 2025-09-28 07:51:16,232 - lava_callback - INFO - Completed processing callback for node 68d8347c1fd44e101337fd75 INFO: 172.21.0.1:33830 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:51:20,561 - lava_callback - INFO - Received callback for node 68d64cc1a5c3ab168ecc79f2 INFO: 172.21.0.1:34834 - "POST /node/68d64cc1a5c3ab168ecc79f2 HTTP/1.0" 202 Accepted 2025-09-28 07:51:20,567 - lava_callback - INFO - Starting async job submit for node 68d64cc1a5c3ab168ecc79f2 2025-09-28 07:51:20,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:20,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc1a5c3ab168ecc79f2 HTTP/1.1" 200 None 2025-09-28 07:51:20,612 - lava_callback - DEBUG - Processing node 68d64cc1a5c3ab168ecc79f2: ltp-input 2025-09-28 07:51:21,022 - lava_callback - ERROR - Error processing callback for node 68d64cc1a5c3ab168ecc79f2: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56896 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:51:43,516 - lava_callback - INFO - Received callback for node 68d8347f1fd44e101337fd78 INFO: 172.21.0.1:46798 - "POST /node/68d8347f1fd44e101337fd78 HTTP/1.0" 202 Accepted 2025-09-28 07:51:43,522 - lava_callback - INFO - Starting async job submit for node 68d8347f1fd44e101337fd78 2025-09-28 07:51:43,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:43,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347f1fd44e101337fd78 HTTP/1.1" 200 None 2025-09-28 07:51:43,584 - lava_callback - DEBUG - Processing node 68d8347f1fd44e101337fd78: baseline-arm64 2025-09-28 07:51:43,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:51:44,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:51:44,030 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347f1fd44e101337fd78/log.txt.gz 2025-09-28 07:51:44,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:51:44,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:51:44,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347f1fd44e101337fd78/lava_callback.json.gz 2025-09-28 07:51:44,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:44,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347f1fd44e101337fd78 HTTP/1.1" 200 None 2025-09-28 07:51:44,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:44,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:51:44,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:51:44,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8347f1fd44e101337fd78 HTTP/1.1" 200 None 2025-09-28 07:51:44,309 - lava_callback - INFO - Completed processing callback for node 68d8347f1fd44e101337fd78 INFO: 172.21.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:52:21,272 - lava_callback - INFO - Received callback for node 68d8347f1fd44e101337fd7b INFO: 172.21.0.1:43826 - "POST /node/68d8347f1fd44e101337fd7b HTTP/1.0" 202 Accepted 2025-09-28 07:52:21,279 - lava_callback - INFO - Starting async job submit for node 68d8347f1fd44e101337fd7b 2025-09-28 07:52:21,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:21,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347f1fd44e101337fd7b HTTP/1.1" 200 None 2025-09-28 07:52:21,312 - lava_callback - DEBUG - Processing node 68d8347f1fd44e101337fd7b: baseline-arm64 2025-09-28 07:52:21,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:52:21,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:52:21,755 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347f1fd44e101337fd7b/log.txt.gz 2025-09-28 07:52:21,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:52:21,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:52:21,895 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d8347f1fd44e101337fd7b/lava_callback.json.gz 2025-09-28 07:52:21,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:21,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8347f1fd44e101337fd7b HTTP/1.1" 200 None 2025-09-28 07:52:21,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:21,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:52:21,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:21,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8347f1fd44e101337fd7b HTTP/1.1" 200 None 2025-09-28 07:52:21,976 - lava_callback - INFO - Completed processing callback for node 68d8347f1fd44e101337fd7b INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:52:33,204 - lava_callback - INFO - Received callback for node 68d8a4687aff7facd9a8d57b INFO: 172.21.0.1:46100 - "POST /node/68d8a4687aff7facd9a8d57b HTTP/1.0" 202 Accepted 2025-09-28 07:52:33,210 - lava_callback - INFO - Starting async job submit for node 68d8a4687aff7facd9a8d57b 2025-09-28 07:52:33,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:33,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a4687aff7facd9a8d57b HTTP/1.1" 200 None 2025-09-28 07:52:33,271 - lava_callback - DEBUG - Processing node 68d8a4687aff7facd9a8d57b: baseline-x86-kcidebug-intel 2025-09-28 07:52:33,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:52:33,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:52:33,627 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d8a4687aff7facd9a8d57b/log.txt.gz 2025-09-28 07:52:33,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:52:33,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:52:33,749 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d8a4687aff7facd9a8d57b/lava_callback.json.gz DEBUG: Setup failed for 68d8a4687aff7facd9a8d57b. Transitting job node result: incomplete -> fail 2025-09-28 07:52:33,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:33,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8a4687aff7facd9a8d57b HTTP/1.1" 200 None 2025-09-28 07:52:33,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:33,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f617aff7facd9a8d155 HTTP/1.1" 200 None 2025-09-28 07:52:33,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:52:33,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8a4687aff7facd9a8d57b HTTP/1.1" 200 None 2025-09-28 07:52:33,870 - lava_callback - INFO - Completed processing callback for node 68d8a4687aff7facd9a8d57b INFO: 172.21.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:53:07,536 - lava_callback - INFO - Received callback for node 68d834821fd44e101337fd7e INFO: 172.21.0.1:40086 - "POST /node/68d834821fd44e101337fd7e HTTP/1.0" 202 Accepted 2025-09-28 07:53:07,543 - lava_callback - INFO - Starting async job submit for node 68d834821fd44e101337fd7e 2025-09-28 07:53:07,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:07,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834821fd44e101337fd7e HTTP/1.1" 200 None 2025-09-28 07:53:07,579 - lava_callback - DEBUG - Processing node 68d834821fd44e101337fd7e: baseline-arm64 2025-09-28 07:53:07,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:53:08,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:53:08,018 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834821fd44e101337fd7e/log.txt.gz 2025-09-28 07:53:08,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:53:08,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:53:08,157 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834821fd44e101337fd7e/lava_callback.json.gz 2025-09-28 07:53:08,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:08,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834821fd44e101337fd7e HTTP/1.1" 200 None 2025-09-28 07:53:08,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:08,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:53:08,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:08,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834821fd44e101337fd7e HTTP/1.1" 200 None 2025-09-28 07:53:08,238 - lava_callback - INFO - Completed processing callback for node 68d834821fd44e101337fd7e INFO: 172.21.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:53:38,802 - lava_callback - INFO - Received callback for node 68d834821fd44e101337fd81 INFO: 172.21.0.1:37766 - "POST /node/68d834821fd44e101337fd81 HTTP/1.0" 202 Accepted 2025-09-28 07:53:38,808 - lava_callback - INFO - Starting async job submit for node 68d834821fd44e101337fd81 2025-09-28 07:53:38,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:38,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834821fd44e101337fd81 HTTP/1.1" 200 None 2025-09-28 07:53:38,849 - lava_callback - DEBUG - Processing node 68d834821fd44e101337fd81: baseline-arm64 2025-09-28 07:53:39,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:53:39,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:53:39,365 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834821fd44e101337fd81/log.txt.gz 2025-09-28 07:53:39,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:53:39,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:53:39,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834821fd44e101337fd81/lava_callback.json.gz 2025-09-28 07:53:39,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:39,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834821fd44e101337fd81 HTTP/1.1" 200 None 2025-09-28 07:53:39,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:39,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:53:39,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:53:39,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834821fd44e101337fd81 HTTP/1.1" 200 None 2025-09-28 07:53:39,612 - lava_callback - INFO - Completed processing callback for node 68d834821fd44e101337fd81 INFO: 172.21.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51650 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:54:24,749 - lava_callback - INFO - Received callback for node 68d834851fd44e101337fd84 INFO: 172.21.0.1:47932 - "POST /node/68d834851fd44e101337fd84 HTTP/1.0" 202 Accepted 2025-09-28 07:54:24,755 - lava_callback - INFO - Starting async job submit for node 68d834851fd44e101337fd84 2025-09-28 07:54:24,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:24,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834851fd44e101337fd84 HTTP/1.1" 200 None 2025-09-28 07:54:24,788 - lava_callback - DEBUG - Processing node 68d834851fd44e101337fd84: baseline-arm64 2025-09-28 07:54:25,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:54:25,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:54:25,206 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834851fd44e101337fd84/log.txt.gz 2025-09-28 07:54:25,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:54:25,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:54:25,367 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834851fd44e101337fd84/lava_callback.json.gz 2025-09-28 07:54:25,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:25,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834851fd44e101337fd84 HTTP/1.1" 200 None 2025-09-28 07:54:25,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:25,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:54:25,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:25,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834851fd44e101337fd84 HTTP/1.1" 200 None 2025-09-28 07:54:25,472 - lava_callback - INFO - Completed processing callback for node 68d834851fd44e101337fd84 INFO: 172.21.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56772 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:54:56,265 - lava_callback - INFO - Received callback for node 68d834861fd44e101337fd87 INFO: 172.21.0.1:38976 - "POST /node/68d834861fd44e101337fd87 HTTP/1.0" 202 Accepted 2025-09-28 07:54:56,271 - lava_callback - INFO - Starting async job submit for node 68d834861fd44e101337fd87 2025-09-28 07:54:56,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:56,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834861fd44e101337fd87 HTTP/1.1" 200 None 2025-09-28 07:54:56,304 - lava_callback - DEBUG - Processing node 68d834861fd44e101337fd87: baseline-arm64 INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:54:56,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:54:56,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:54:56,761 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834861fd44e101337fd87/log.txt.gz 2025-09-28 07:54:56,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:54:56,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:54:56,900 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834861fd44e101337fd87/lava_callback.json.gz 2025-09-28 07:54:56,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:56,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834861fd44e101337fd87 HTTP/1.1" 200 None 2025-09-28 07:54:56,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:56,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72e52a6a7b6615d4c4172 HTTP/1.1" 200 None 2025-09-28 07:54:56,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:54:56,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834861fd44e101337fd87 HTTP/1.1" 200 None 2025-09-28 07:54:57,000 - lava_callback - INFO - Completed processing callback for node 68d834861fd44e101337fd87 INFO: 172.21.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47166 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:55:20,476 - lava_callback - INFO - Received callback for node 68d64cc1a5c3ab168ecc79f7 INFO: 172.21.0.1:42928 - "POST /node/68d64cc1a5c3ab168ecc79f7 HTTP/1.0" 202 Accepted 2025-09-28 07:55:20,483 - lava_callback - INFO - Starting async job submit for node 68d64cc1a5c3ab168ecc79f7 2025-09-28 07:55:20,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:20,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc1a5c3ab168ecc79f7 HTTP/1.1" 200 None 2025-09-28 07:55:20,551 - lava_callback - DEBUG - Processing node 68d64cc1a5c3ab168ecc79f7: ltp-ipc 2025-09-28 07:55:20,881 - lava_callback - ERROR - Error processing callback for node 68d64cc1a5c3ab168ecc79f7: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:55:45,207 - lava_callback - INFO - Received callback for node 68d834c51fd44e101337fdba INFO: 172.21.0.1:40868 - "POST /node/68d834c51fd44e101337fdba HTTP/1.0" 202 Accepted 2025-09-28 07:55:45,213 - lava_callback - INFO - Starting async job submit for node 68d834c51fd44e101337fdba 2025-09-28 07:55:45,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:45,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c51fd44e101337fdba HTTP/1.1" 200 None 2025-09-28 07:55:45,255 - lava_callback - DEBUG - Processing node 68d834c51fd44e101337fdba: baseline-arm64 2025-09-28 07:55:45,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:55:45,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:55:45,754 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c51fd44e101337fdba/log.txt.gz 2025-09-28 07:55:45,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:55:45,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:55:45,904 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c51fd44e101337fdba/lava_callback.json.gz 2025-09-28 07:55:45,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:45,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c51fd44e101337fdba HTTP/1.1" 200 None 2025-09-28 07:55:45,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:45,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 07:55:45,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:46,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834c51fd44e101337fdba HTTP/1.1" 200 None 2025-09-28 07:55:46,002 - lava_callback - INFO - Completed processing callback for node 68d834c51fd44e101337fdba INFO: 172.21.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53400 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:55:56,543 - lava_callback - INFO - Received callback for node 68d834c51fd44e101337fdbd INFO: 172.21.0.1:53416 - "POST /node/68d834c51fd44e101337fdbd HTTP/1.0" 202 Accepted 2025-09-28 07:55:56,549 - lava_callback - INFO - Starting async job submit for node 68d834c51fd44e101337fdbd 2025-09-28 07:55:56,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:56,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c51fd44e101337fdbd HTTP/1.1" 200 None 2025-09-28 07:55:56,583 - lava_callback - DEBUG - Processing node 68d834c51fd44e101337fdbd: baseline-arm64 2025-09-28 07:55:56,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:55:57,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:55:57,061 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c51fd44e101337fdbd/log.txt.gz 2025-09-28 07:55:57,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:55:57,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:55:57,218 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c51fd44e101337fdbd/lava_callback.json.gz 2025-09-28 07:55:57,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:57,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c51fd44e101337fdbd HTTP/1.1" 200 None 2025-09-28 07:55:57,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:57,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 07:55:57,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:55:57,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834c51fd44e101337fdbd HTTP/1.1" 200 None 2025-09-28 07:55:57,297 - lava_callback - INFO - Completed processing callback for node 68d834c51fd44e101337fdbd INFO: 172.21.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:56:29,856 - lava_callback - INFO - Received callback for node 68d64cbfa5c3ab168ecc79e5 INFO: 172.21.0.1:44368 - "POST /node/68d64cbfa5c3ab168ecc79e5 HTTP/1.0" 202 Accepted 2025-09-28 07:56:29,864 - lava_callback - INFO - Starting async job submit for node 68d64cbfa5c3ab168ecc79e5 2025-09-28 07:56:29,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:56:29,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cbfa5c3ab168ecc79e5 HTTP/1.1" 200 None 2025-09-28 07:56:29,983 - lava_callback - DEBUG - Processing node 68d64cbfa5c3ab168ecc79e5: ltp-cve 2025-09-28 07:56:30,706 - lava_callback - ERROR - Error processing callback for node 68d64cbfa5c3ab168ecc79e5: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:44372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:57:05,229 - lava_callback - INFO - Received callback for node 68d834c81fd44e101337fdc0 INFO: 172.21.0.1:35286 - "POST /node/68d834c81fd44e101337fdc0 HTTP/1.0" 202 Accepted 2025-09-28 07:57:05,235 - lava_callback - INFO - Starting async job submit for node 68d834c81fd44e101337fdc0 2025-09-28 07:57:05,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:05,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c81fd44e101337fdc0 HTTP/1.1" 200 None 2025-09-28 07:57:05,292 - lava_callback - DEBUG - Processing node 68d834c81fd44e101337fdc0: baseline-arm64 2025-09-28 07:57:05,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:57:05,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:57:05,777 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c81fd44e101337fdc0/log.txt.gz 2025-09-28 07:57:05,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:57:05,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:57:05,936 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c81fd44e101337fdc0/lava_callback.json.gz 2025-09-28 07:57:05,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:05,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c81fd44e101337fdc0 HTTP/1.1" 200 None 2025-09-28 07:57:05,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:05,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 07:57:05,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:06,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834c81fd44e101337fdc0 HTTP/1.1" 200 None 2025-09-28 07:57:06,013 - lava_callback - INFO - Completed processing callback for node 68d834c81fd44e101337fdc0 INFO: 172.21.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56760 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:57:16,640 - lava_callback - INFO - Received callback for node 68d834c91fd44e101337fdc3 INFO: 172.21.0.1:56766 - "POST /node/68d834c91fd44e101337fdc3 HTTP/1.0" 202 Accepted 2025-09-28 07:57:16,646 - lava_callback - INFO - Starting async job submit for node 68d834c91fd44e101337fdc3 2025-09-28 07:57:16,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:16,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c91fd44e101337fdc3 HTTP/1.1" 200 None 2025-09-28 07:57:16,678 - lava_callback - DEBUG - Processing node 68d834c91fd44e101337fdc3: baseline-arm64 2025-09-28 07:57:17,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:57:17,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:57:17,186 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c91fd44e101337fdc3/log.txt.gz 2025-09-28 07:57:17,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:57:17,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:57:17,323 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834c91fd44e101337fdc3/lava_callback.json.gz 2025-09-28 07:57:17,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:17,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834c91fd44e101337fdc3 HTTP/1.1" 200 None 2025-09-28 07:57:17,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:17,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 07:57:17,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:57:17,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834c91fd44e101337fdc3 HTTP/1.1" 200 None 2025-09-28 07:57:17,398 - lava_callback - INFO - Completed processing callback for node 68d834c91fd44e101337fdc3 INFO: 172.21.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:58:54,833 - lava_callback - INFO - Received callback for node 68d834cb1fd44e101337fdc6 INFO: 172.21.0.1:45212 - "POST /node/68d834cb1fd44e101337fdc6 HTTP/1.0" 202 Accepted 2025-09-28 07:58:54,840 - lava_callback - INFO - Starting async job submit for node 68d834cb1fd44e101337fdc6 2025-09-28 07:58:54,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:58:54,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834cb1fd44e101337fdc6 HTTP/1.1" 200 None 2025-09-28 07:58:54,875 - lava_callback - DEBUG - Processing node 68d834cb1fd44e101337fdc6: baseline-arm64 2025-09-28 07:58:55,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:58:55,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:58:55,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834cb1fd44e101337fdc6/log.txt.gz 2025-09-28 07:58:55,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:58:55,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:58:55,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834cb1fd44e101337fdc6/lava_callback.json.gz 2025-09-28 07:58:55,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:58:55,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834cb1fd44e101337fdc6 HTTP/1.1" 200 None 2025-09-28 07:58:55,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:58:55,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 07:58:55,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:58:55,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834cb1fd44e101337fdc6 HTTP/1.1" 200 None 2025-09-28 07:58:55,581 - lava_callback - INFO - Completed processing callback for node 68d834cb1fd44e101337fdc6 INFO: 172.21.0.1:45214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:59:10,465 - lava_callback - INFO - Received callback for node 68d834cc1fd44e101337fdc9 INFO: 172.21.0.1:60438 - "POST /node/68d834cc1fd44e101337fdc9 HTTP/1.0" 202 Accepted 2025-09-28 07:59:10,472 - lava_callback - INFO - Starting async job submit for node 68d834cc1fd44e101337fdc9 2025-09-28 07:59:10,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:10,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834cc1fd44e101337fdc9 HTTP/1.1" 200 None 2025-09-28 07:59:10,535 - lava_callback - DEBUG - Processing node 68d834cc1fd44e101337fdc9: baseline-arm64 2025-09-28 07:59:10,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:59:11,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:59:11,053 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834cc1fd44e101337fdc9/log.txt.gz 2025-09-28 07:59:11,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:59:11,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:59:11,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834cc1fd44e101337fdc9/lava_callback.json.gz 2025-09-28 07:59:11,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:11,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834cc1fd44e101337fdc9 HTTP/1.1" 200 None 2025-09-28 07:59:11,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:11,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 07:59:11,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:11,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834cc1fd44e101337fdc9 HTTP/1.1" 200 None 2025-09-28 07:59:11,348 - lava_callback - INFO - Completed processing callback for node 68d834cc1fd44e101337fdc9 INFO: 172.21.0.1:60450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60464 - "GET / HTTP/1.0" 200 OK 2025-09-28 07:59:20,067 - lava_callback - INFO - Received callback for node 68d81b221fd44e101337f79b INFO: 172.21.0.1:33352 - "POST /node/68d81b221fd44e101337f79b HTTP/1.0" 202 Accepted 2025-09-28 07:59:20,074 - lava_callback - INFO - Starting async job submit for node 68d81b221fd44e101337f79b 2025-09-28 07:59:20,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:20,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b221fd44e101337f79b HTTP/1.1" 200 None 2025-09-28 07:59:20,128 - lava_callback - DEBUG - Processing node 68d81b221fd44e101337f79b: baseline-arm64 2025-09-28 07:59:21,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:59:21,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:59:21,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b221fd44e101337f79b/log.txt.gz 2025-09-28 07:59:21,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 07:59:21,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 07:59:21,319 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d81b221fd44e101337f79b/lava_callback.json.gz DEBUG: Setup failed for 68d81b221fd44e101337f79b. Transitting job node result: incomplete -> fail 2025-09-28 07:59:21,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:21,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d81b221fd44e101337f79b HTTP/1.1" 200 None 2025-09-28 07:59:21,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:21,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d651ddcca86fb17472aa24 HTTP/1.1" 200 None 2025-09-28 07:59:21,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 07:59:21,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d81b221fd44e101337f79b HTTP/1.1" 200 None 2025-09-28 07:59:21,460 - lava_callback - INFO - Completed processing callback for node 68d81b221fd44e101337f79b INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44738 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:00:08,118 - lava_callback - INFO - Received callback for node 68d834ce1fd44e101337fdcc INFO: 172.21.0.1:44752 - "POST /node/68d834ce1fd44e101337fdcc HTTP/1.0" 202 Accepted 2025-09-28 08:00:08,124 - lava_callback - INFO - Starting async job submit for node 68d834ce1fd44e101337fdcc 2025-09-28 08:00:08,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:08,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834ce1fd44e101337fdcc HTTP/1.1" 200 None 2025-09-28 08:00:08,157 - lava_callback - DEBUG - Processing node 68d834ce1fd44e101337fdcc: baseline-arm64 2025-09-28 08:00:08,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:08,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:08,647 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834ce1fd44e101337fdcc/log.txt.gz 2025-09-28 08:00:08,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:08,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:08,780 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834ce1fd44e101337fdcc/lava_callback.json.gz 2025-09-28 08:00:08,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:08,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834ce1fd44e101337fdcc HTTP/1.1" 200 None 2025-09-28 08:00:08,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:08,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:00:08,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:08,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834ce1fd44e101337fdcc HTTP/1.1" 200 None 2025-09-28 08:00:08,863 - lava_callback - INFO - Completed processing callback for node 68d834ce1fd44e101337fdcc INFO: 172.21.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:00:16,747 - lava_callback - INFO - Received callback for node 68d834cf1fd44e101337fdcf INFO: 172.21.0.1:41858 - "POST /node/68d834cf1fd44e101337fdcf HTTP/1.0" 202 Accepted 2025-09-28 08:00:16,753 - lava_callback - INFO - Starting async job submit for node 68d834cf1fd44e101337fdcf 2025-09-28 08:00:16,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:16,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834cf1fd44e101337fdcf HTTP/1.1" 200 None 2025-09-28 08:00:16,789 - lava_callback - DEBUG - Processing node 68d834cf1fd44e101337fdcf: baseline-arm64 2025-09-28 08:00:17,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:17,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:17,275 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834cf1fd44e101337fdcf/log.txt.gz 2025-09-28 08:00:17,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:17,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:17,396 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834cf1fd44e101337fdcf/lava_callback.json.gz 2025-09-28 08:00:17,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:17,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834cf1fd44e101337fdcf HTTP/1.1" 200 None 2025-09-28 08:00:17,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:17,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:00:17,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:17,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834cf1fd44e101337fdcf HTTP/1.1" 200 None 2025-09-28 08:00:17,489 - lava_callback - INFO - Completed processing callback for node 68d834cf1fd44e101337fdcf INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:00:27,808 - lava_callback - INFO - Received callback for node 68d8c63ca178a5bbc0ea6e3b INFO: 172.21.0.1:43292 - "POST /node/68d8c63ca178a5bbc0ea6e3b HTTP/1.0" 202 Accepted 2025-09-28 08:00:27,814 - lava_callback - INFO - Starting async job submit for node 68d8c63ca178a5bbc0ea6e3b 2025-09-28 08:00:27,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:27,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c63ca178a5bbc0ea6e3b HTTP/1.1" 200 None 2025-09-28 08:00:27,855 - lava_callback - DEBUG - Processing node 68d8c63ca178a5bbc0ea6e3b: baseline-arm 2025-09-28 08:00:28,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:28,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:28,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-68d8c63ca178a5bbc0ea6e3b/log.txt.gz 2025-09-28 08:00:28,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:28,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:28,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-68d8c63ca178a5bbc0ea6e3b/lava_callback.json.gz DEBUG: Setup failed for 68d8c63ca178a5bbc0ea6e3b. Transitting job node result: incomplete -> fail 2025-09-28 08:00:28,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:28,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8c63ca178a5bbc0ea6e3b HTTP/1.1" 200 None 2025-09-28 08:00:28,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:28,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f347aff7facd9a8d105 HTTP/1.1" 200 None 2025-09-28 08:00:28,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:28,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8c63ca178a5bbc0ea6e3b HTTP/1.1" 200 None 2025-09-28 08:00:28,492 - lava_callback - INFO - Completed processing callback for node 68d8c63ca178a5bbc0ea6e3b INFO: 172.21.0.1:55126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42808 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:00:47,072 - lava_callback - INFO - Received callback for node 68d834d11fd44e101337fdd2 INFO: 172.21.0.1:42818 - "POST /node/68d834d11fd44e101337fdd2 HTTP/1.0" 202 Accepted 2025-09-28 08:00:47,078 - lava_callback - INFO - Starting async job submit for node 68d834d11fd44e101337fdd2 2025-09-28 08:00:47,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:47,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d11fd44e101337fdd2 HTTP/1.1" 200 None 2025-09-28 08:00:47,111 - lava_callback - DEBUG - Processing node 68d834d11fd44e101337fdd2: baseline-arm64 2025-09-28 08:00:47,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:47,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:47,630 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d11fd44e101337fdd2/log.txt.gz 2025-09-28 08:00:47,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:00:47,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:00:47,749 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d11fd44e101337fdd2/lava_callback.json.gz 2025-09-28 08:00:47,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:47,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d11fd44e101337fdd2 HTTP/1.1" 200 None 2025-09-28 08:00:47,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:47,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:00:47,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:00:47,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834d11fd44e101337fdd2 HTTP/1.1" 200 None 2025-09-28 08:00:47,837 - lava_callback - INFO - Completed processing callback for node 68d834d11fd44e101337fdd2 INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59846 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:01:00,918 - lava_callback - INFO - Received callback for node 68d64cc2a5c3ab168ecc79fb INFO: 172.21.0.1:59850 - "POST /node/68d64cc2a5c3ab168ecc79fb HTTP/1.0" 202 Accepted 2025-09-28 08:01:00,925 - lava_callback - INFO - Starting async job submit for node 68d64cc2a5c3ab168ecc79fb 2025-09-28 08:01:00,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:00,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc2a5c3ab168ecc79fb HTTP/1.1" 200 None 2025-09-28 08:01:00,977 - lava_callback - DEBUG - Processing node 68d64cc2a5c3ab168ecc79fb: ltp-pty 2025-09-28 08:01:01,351 - lava_callback - ERROR - Error processing callback for node 68d64cc2a5c3ab168ecc79fb: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:59860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39766 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:01:27,756 - lava_callback - INFO - Received callback for node 68d834d21fd44e101337fdd5 INFO: 172.21.0.1:39776 - "POST /node/68d834d21fd44e101337fdd5 HTTP/1.0" 202 Accepted 2025-09-28 08:01:27,762 - lava_callback - INFO - Starting async job submit for node 68d834d21fd44e101337fdd5 2025-09-28 08:01:27,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:27,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d21fd44e101337fdd5 HTTP/1.1" 200 None 2025-09-28 08:01:27,795 - lava_callback - DEBUG - Processing node 68d834d21fd44e101337fdd5: baseline-arm64 2025-09-28 08:01:28,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:01:28,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:01:28,307 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d21fd44e101337fdd5/log.txt.gz 2025-09-28 08:01:28,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:01:28,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:01:28,449 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d21fd44e101337fdd5/lava_callback.json.gz 2025-09-28 08:01:28,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:28,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d21fd44e101337fdd5 HTTP/1.1" 200 None 2025-09-28 08:01:28,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:28,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:01:28,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:28,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834d21fd44e101337fdd5 HTTP/1.1" 200 None 2025-09-28 08:01:28,524 - lava_callback - INFO - Completed processing callback for node 68d834d21fd44e101337fdd5 INFO: 172.21.0.1:60602 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:01:36,729 - lava_callback - INFO - Received callback for node 68d834d41fd44e101337fdd8 INFO: 172.21.0.1:60604 - "POST /node/68d834d41fd44e101337fdd8 HTTP/1.0" 202 Accepted 2025-09-28 08:01:36,735 - lava_callback - INFO - Starting async job submit for node 68d834d41fd44e101337fdd8 2025-09-28 08:01:36,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:36,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d41fd44e101337fdd8 HTTP/1.1" 200 None 2025-09-28 08:01:36,769 - lava_callback - DEBUG - Processing node 68d834d41fd44e101337fdd8: baseline-arm64 2025-09-28 08:01:37,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:01:37,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:01:37,236 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d41fd44e101337fdd8/log.txt.gz 2025-09-28 08:01:37,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:01:37,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:01:37,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d41fd44e101337fdd8/lava_callback.json.gz 2025-09-28 08:01:37,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:37,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d41fd44e101337fdd8 HTTP/1.1" 200 None 2025-09-28 08:01:37,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:37,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:01:37,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:01:37,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834d41fd44e101337fdd8 HTTP/1.1" 200 None 2025-09-28 08:01:37,437 - lava_callback - INFO - Completed processing callback for node 68d834d41fd44e101337fdd8 INFO: 172.21.0.1:46348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57146 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:02:08,865 - lava_callback - INFO - Received callback for node 68d834d51fd44e101337fddb INFO: 172.21.0.1:57158 - "POST /node/68d834d51fd44e101337fddb HTTP/1.0" 202 Accepted 2025-09-28 08:02:08,871 - lava_callback - INFO - Starting async job submit for node 68d834d51fd44e101337fddb 2025-09-28 08:02:08,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:08,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d51fd44e101337fddb HTTP/1.1" 200 None 2025-09-28 08:02:08,906 - lava_callback - DEBUG - Processing node 68d834d51fd44e101337fddb: baseline-arm64 2025-09-28 08:02:09,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:02:09,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:02:09,429 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d51fd44e101337fddb/log.txt.gz 2025-09-28 08:02:09,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:02:09,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:02:09,550 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d51fd44e101337fddb/lava_callback.json.gz 2025-09-28 08:02:09,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:09,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d51fd44e101337fddb HTTP/1.1" 200 None 2025-09-28 08:02:09,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:09,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:02:09,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:09,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834d51fd44e101337fddb HTTP/1.1" 200 None 2025-09-28 08:02:09,657 - lava_callback - INFO - Completed processing callback for node 68d834d51fd44e101337fddb INFO: 172.21.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:02:45,311 - lava_callback - INFO - Received callback for node 68d834d81fd44e101337fde1 INFO: 172.21.0.1:47882 - "POST /node/68d834d81fd44e101337fde1 HTTP/1.0" 202 Accepted 2025-09-28 08:02:45,319 - lava_callback - INFO - Starting async job submit for node 68d834d81fd44e101337fde1 2025-09-28 08:02:45,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:45,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d81fd44e101337fde1 HTTP/1.1" 200 None 2025-09-28 08:02:45,359 - lava_callback - DEBUG - Processing node 68d834d81fd44e101337fde1: baseline-arm64 2025-09-28 08:02:45,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:02:45,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:02:45,938 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d81fd44e101337fde1/log.txt.gz 2025-09-28 08:02:45,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:02:46,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:02:46,092 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d81fd44e101337fde1/lava_callback.json.gz 2025-09-28 08:02:46,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:46,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d81fd44e101337fde1 HTTP/1.1" 200 None 2025-09-28 08:02:46,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:46,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:02:46,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:46,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834d81fd44e101337fde1 HTTP/1.1" 200 None 2025-09-28 08:02:46,198 - lava_callback - INFO - Completed processing callback for node 68d834d81fd44e101337fde1 INFO: 172.21.0.1:47896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46284 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:02:57,090 - lava_callback - INFO - Received callback for node 68d834d71fd44e101337fdde INFO: 172.21.0.1:46292 - "POST /node/68d834d71fd44e101337fdde HTTP/1.0" 202 Accepted 2025-09-28 08:02:57,096 - lava_callback - INFO - Starting async job submit for node 68d834d71fd44e101337fdde 2025-09-28 08:02:57,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:57,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d71fd44e101337fdde HTTP/1.1" 200 None 2025-09-28 08:02:57,138 - lava_callback - DEBUG - Processing node 68d834d71fd44e101337fdde: baseline-arm64 2025-09-28 08:02:57,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:02:57,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:02:57,630 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d71fd44e101337fdde/log.txt.gz 2025-09-28 08:02:57,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:02:57,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:02:57,798 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834d71fd44e101337fdde/lava_callback.json.gz 2025-09-28 08:02:57,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:57,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834d71fd44e101337fdde HTTP/1.1" 200 None 2025-09-28 08:02:57,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:57,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:02:57,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:02:57,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834d71fd44e101337fdde HTTP/1.1" 200 None 2025-09-28 08:02:57,886 - lava_callback - INFO - Completed processing callback for node 68d834d71fd44e101337fdde INFO: 172.21.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47260 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:03:26,365 - lava_callback - INFO - Received callback for node 68d834da1fd44e101337fde4 INFO: 172.21.0.1:39424 - "POST /node/68d834da1fd44e101337fde4 HTTP/1.0" 202 Accepted 2025-09-28 08:03:26,372 - lava_callback - INFO - Starting async job submit for node 68d834da1fd44e101337fde4 2025-09-28 08:03:26,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:26,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834da1fd44e101337fde4 HTTP/1.1" 200 None 2025-09-28 08:03:26,433 - lava_callback - DEBUG - Processing node 68d834da1fd44e101337fde4: baseline-arm64 INFO: 172.21.0.1:39436 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:03:26,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:03:26,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:03:26,892 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834da1fd44e101337fde4/log.txt.gz 2025-09-28 08:03:26,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:03:27,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:03:27,101 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834da1fd44e101337fde4/lava_callback.json.gz 2025-09-28 08:03:27,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:27,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834da1fd44e101337fde4 HTTP/1.1" 200 None 2025-09-28 08:03:27,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:27,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:03:27,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:27,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834da1fd44e101337fde4 HTTP/1.1" 200 None 2025-09-28 08:03:27,215 - lava_callback - INFO - Completed processing callback for node 68d834da1fd44e101337fde4 INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:03:56,678 - lava_callback - INFO - Received callback for node 68d834db1fd44e101337fde7 INFO: 172.21.0.1:43814 - "POST /node/68d834db1fd44e101337fde7 HTTP/1.0" 202 Accepted 2025-09-28 08:03:56,688 - lava_callback - INFO - Starting async job submit for node 68d834db1fd44e101337fde7 2025-09-28 08:03:56,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:56,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834db1fd44e101337fde7 HTTP/1.1" 200 None 2025-09-28 08:03:56,742 - lava_callback - DEBUG - Processing node 68d834db1fd44e101337fde7: baseline-arm64 2025-09-28 08:03:57,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:03:57,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:03:57,527 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834db1fd44e101337fde7/log.txt.gz 2025-09-28 08:03:57,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:03:57,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:03:57,723 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-68d834db1fd44e101337fde7/lava_callback.json.gz 2025-09-28 08:03:57,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:57,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d834db1fd44e101337fde7 HTTP/1.1" 200 None 2025-09-28 08:03:57,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:57,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d72a303c94e462c68828aa HTTP/1.1" 200 None 2025-09-28 08:03:57,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:03:57,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d834db1fd44e101337fde7 HTTP/1.1" 200 None 2025-09-28 08:03:57,832 - lava_callback - INFO - Completed processing callback for node 68d834db1fd44e101337fde7 INFO: 172.21.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54444 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:05:10,715 - lava_callback - INFO - Received callback for node 68d64cc3a5c3ab168ecc7a01 INFO: 172.21.0.1:35826 - "POST /node/68d64cc3a5c3ab168ecc7a01 HTTP/1.0" 202 Accepted 2025-09-28 08:05:10,727 - lava_callback - INFO - Starting async job submit for node 68d64cc3a5c3ab168ecc7a01 2025-09-28 08:05:10,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:05:10,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc3a5c3ab168ecc7a01 HTTP/1.1" 200 None 2025-09-28 08:05:10,825 - lava_callback - DEBUG - Processing node 68d64cc3a5c3ab168ecc7a01: ltp-smoketest 2025-09-28 08:05:11,533 - lava_callback - ERROR - Error processing callback for node 68d64cc3a5c3ab168ecc7a01: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:35836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:09:15,005 - lava_callback - INFO - Received callback for node 68d64cc5a5c3ab168ecc7a0d INFO: 172.21.0.1:45686 - "POST /node/68d64cc5a5c3ab168ecc7a0d HTTP/1.0" 202 Accepted 2025-09-28 08:09:15,012 - lava_callback - INFO - Starting async job submit for node 68d64cc5a5c3ab168ecc7a0d 2025-09-28 08:09:15,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:09:15,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc5a5c3ab168ecc7a0d HTTP/1.1" 200 None 2025-09-28 08:09:15,080 - lava_callback - DEBUG - Processing node 68d64cc5a5c3ab168ecc7a0d: ltp-timers INFO: 172.21.0.1:45694 - "GET /api/metrics HTTP/1.0" 200 OK 2025-09-28 08:09:15,412 - lava_callback - ERROR - Error processing callback for node 68d64cc5a5c3ab168ecc7a0d: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:45700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45438 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:12:45,408 - lava_callback - INFO - Received callback for node 68d8e942a178a5bbc0ea710f INFO: 172.21.0.1:35150 - "POST /node/68d8e942a178a5bbc0ea710f HTTP/1.0" 202 Accepted 2025-09-28 08:12:45,414 - lava_callback - INFO - Starting async job submit for node 68d8e942a178a5bbc0ea710f 2025-09-28 08:12:45,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:12:45,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8e942a178a5bbc0ea710f HTTP/1.1" 200 None 2025-09-28 08:12:45,490 - lava_callback - DEBUG - Processing node 68d8e942a178a5bbc0ea710f: baseline-x86-kcidebug-intel 2025-09-28 08:12:45,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:12:45,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:12:45,888 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d8e942a178a5bbc0ea710f/log.txt.gz 2025-09-28 08:12:45,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:12:46,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:12:46,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-kcidebug-intel-68d8e942a178a5bbc0ea710f/lava_callback.json.gz DEBUG: Setup failed for 68d8e942a178a5bbc0ea710f. Transitting job node result: incomplete -> fail 2025-09-28 08:12:46,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:12:46,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8e942a178a5bbc0ea710f HTTP/1.1" 200 None 2025-09-28 08:12:46,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:12:46,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d89f617aff7facd9a8d155 HTTP/1.1" 200 None 2025-09-28 08:12:46,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:12:46,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8e942a178a5bbc0ea710f HTTP/1.1" 200 None 2025-09-28 08:12:46,189 - lava_callback - INFO - Completed processing callback for node 68d8e942a178a5bbc0ea710f INFO: 172.21.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39990 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:13:53,809 - lava_callback - INFO - Received callback for node 68d64cc5a5c3ab168ecc7a10 INFO: 172.21.0.1:58408 - "POST /node/68d64cc5a5c3ab168ecc7a10 HTTP/1.0" 202 Accepted 2025-09-28 08:13:53,816 - lava_callback - INFO - Starting async job submit for node 68d64cc5a5c3ab168ecc7a10 2025-09-28 08:13:53,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:13:53,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc5a5c3ab168ecc7a10 HTTP/1.1" 200 None 2025-09-28 08:13:53,873 - lava_callback - DEBUG - Processing node 68d64cc5a5c3ab168ecc7a10: ltp-watchqueue 2025-09-28 08:13:54,255 - lava_callback - ERROR - Error processing callback for node 68d64cc5a5c3ab168ecc7a10: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:58416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51510 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:14:21,356 - lava_callback - INFO - Received callback for node 68d64cc3a5c3ab168ecc79fe INFO: 172.21.0.1:43120 - "POST /node/68d64cc3a5c3ab168ecc79fe HTTP/1.0" 202 Accepted 2025-09-28 08:14:21,363 - lava_callback - INFO - Starting async job submit for node 68d64cc3a5c3ab168ecc79fe 2025-09-28 08:14:21,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:14:21,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d64cc3a5c3ab168ecc79fe HTTP/1.1" 200 None 2025-09-28 08:14:21,406 - lava_callback - DEBUG - Processing node 68d64cc3a5c3ab168ecc79fe: ltp-sched 2025-09-28 08:14:21,757 - lava_callback - ERROR - Error processing callback for node 68d64cc3a5c3ab168ecc79fe: None Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 280, in async_job_submit storage = _get_storage(storage_config_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/lava_callback.py", line 172, in _get_storage storage_config = CONFIGS['storage_configs'][storage_config_name] ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: None INFO: 172.21.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51078 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:15:52,038 - lava_callback - INFO - Received callback for node 68d8ee56a178a5bbc0ea7467 INFO: 172.21.0.1:33472 - "POST /node/68d8ee56a178a5bbc0ea7467 HTTP/1.0" 202 Accepted 2025-09-28 08:15:52,047 - lava_callback - INFO - Starting async job submit for node 68d8ee56a178a5bbc0ea7467 2025-09-28 08:15:52,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:15:52,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ee56a178a5bbc0ea7467 HTTP/1.1" 200 None 2025-09-28 08:15:52,096 - lava_callback - DEBUG - Processing node 68d8ee56a178a5bbc0ea7467: baseline-riscv-collabora 2025-09-28 08:15:52,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:15:52,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:15:52,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-riscv-collabora-68d8ee56a178a5bbc0ea7467/log.txt.gz 2025-09-28 08:15:52,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:15:52,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:15:52,698 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-riscv-collabora-68d8ee56a178a5bbc0ea7467/lava_callback.json.gz 2025-09-28 08:15:52,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:15:52,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ee56a178a5bbc0ea7467 HTTP/1.1" 200 None 2025-09-28 08:15:52,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:15:52,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ebe3a178a5bbc0ea7191 HTTP/1.1" 200 None 2025-09-28 08:15:52,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:15:52,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8ee56a178a5bbc0ea7467 HTTP/1.1" 200 None 2025-09-28 08:15:52,832 - lava_callback - INFO - Completed processing callback for node 68d8ee56a178a5bbc0ea7467 INFO: 172.21.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36188 - "GET / HTTP/1.0" 200 OK 2025-09-28 08:16:09,924 - lava_callback - INFO - Received callback for node 68d8ee55a178a5bbc0ea7464 INFO: 172.21.0.1:42428 - "POST /node/68d8ee55a178a5bbc0ea7464 HTTP/1.0" 202 Accepted 2025-09-28 08:16:09,930 - lava_callback - INFO - Starting async job submit for node 68d8ee55a178a5bbc0ea7464 2025-09-28 08:16:09,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:16:09,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ee55a178a5bbc0ea7464 HTTP/1.1" 200 None 2025-09-28 08:16:09,966 - lava_callback - DEBUG - Processing node 68d8ee55a178a5bbc0ea7464: baseline-riscv-broonie 2025-09-28 08:16:10,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:16:10,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:16:10,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-riscv-broonie-68d8ee55a178a5bbc0ea7464/log.txt.gz 2025-09-28 08:16:10,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2025-09-28 08:16:10,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2025-09-28 08:16:10,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-riscv-broonie-68d8ee55a178a5bbc0ea7464/lava_callback.json.gz 2025-09-28 08:16:10,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:16:10,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ee55a178a5bbc0ea7464 HTTP/1.1" 200 None 2025-09-28 08:16:10,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:16:10,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/68d8ebe3a178a5bbc0ea7191 HTTP/1.1" 200 None 2025-09-28 08:16:10,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2025-09-28 08:16:10,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/68d8ee55a178a5bbc0ea7464 HTTP/1.1" 200 None 2025-09-28 08:16:10,591 - lava_callback - INFO - Completed processing callback for node 68d8ee55a178a5bbc0ea7464 INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]