01/14/2026 01:58:18 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 01:58:19 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 01:58:19 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 01:58:19 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:20 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 01:58:20 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 01:58:20 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 01:58:20 PM UTC [DEBUG] Node channel sub id: 102826 01/14/2026 01:58:20 PM UTC [DEBUG] Retry channel sub id: 102827 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 13:58:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:11] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:42] "GET /health HTTP/1.1" 200 - 01/14/2026 02:01:00 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:03 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:04 PM UTC [DEBUG] Job node created: 6967a1a049c7a262599a9397. Parent: 695fb0b4258e18ba90402526 Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:01:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:01:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:43] "GET /health HTTP/1.1" 200 - 01/14/2026 02:10:50 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 616s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:50 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:10:50 PM UTC [ERROR] Thread: scheduler-retry, stuck for 616s 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 01:58:18 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 01:58:19 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 01:58:19 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 01:58:19 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:20 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 01:58:20 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 01:58:20 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 01:58:20 PM UTC [DEBUG] Node channel sub id: 102826 01/14/2026 01:58:20 PM UTC [DEBUG] Retry channel sub id: 102827 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 13:58:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:11] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:42] "GET /health HTTP/1.1" 200 - 01/14/2026 02:01:00 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:03 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:04 PM UTC [DEBUG] Job node created: 6967a1a049c7a262599a9397. Parent: 695fb0b4258e18ba90402526 Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:01:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:01:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:43] "GET /health HTTP/1.1" 200 - 01/14/2026 02:10:50 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 616s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:50 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:10:50 PM UTC [ERROR] Thread: scheduler-retry, stuck for 616s 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:51 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:10:52 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:10:52 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:10:52 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:10:52 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:10:52 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:10:52 PM UTC [DEBUG] Node channel sub id: 102835 01/14/2026 02:10:52 PM UTC [DEBUG] Retry channel sub id: 102836 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:11:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:11:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:12:20] "GET /health HTTP/1.1" 200 - 01/14/2026 02:12:20 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:23 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:28 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:30 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=k3-j721e-sk has no online devices 01/14/2026 02:12:33 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-axg-s400 has no online devices 01/14/2026 02:12:34 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-gxl-s905d-p230 has no online devices Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 01/14/2026 02:12:35 PM UTC [DEBUG] Job node created: 6967a45349c7a262599a93b1. Parent: 695fb0c7258e18ba90402535 127.0.0.1 - - [14/Jan/2026 14:12:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:51] "GET /health HTTP/1.1" 200 - 01/14/2026 02:21:52 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 603s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 02:21:52 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:21:52 PM UTC [ERROR] Thread: scheduler-retry, stuck for 603s 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 01:58:18 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 01:58:19 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 01:58:19 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 01:58:19 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:20 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 01:58:20 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 01:58:20 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 01:58:20 PM UTC [DEBUG] Node channel sub id: 102826 01/14/2026 01:58:20 PM UTC [DEBUG] Retry channel sub id: 102827 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 13:58:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:11] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:42] "GET /health HTTP/1.1" 200 - 01/14/2026 02:01:00 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:03 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:04 PM UTC [DEBUG] Job node created: 6967a1a049c7a262599a9397. Parent: 695fb0b4258e18ba90402526 Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:01:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:01:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:43] "GET /health HTTP/1.1" 200 - 01/14/2026 02:10:50 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 616s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:50 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:10:50 PM UTC [ERROR] Thread: scheduler-retry, stuck for 616s 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:51 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:10:52 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:10:52 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:10:52 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:10:52 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:10:52 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:10:52 PM UTC [DEBUG] Node channel sub id: 102835 01/14/2026 02:10:52 PM UTC [DEBUG] Retry channel sub id: 102836 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:11:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:11:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:12:20] "GET /health HTTP/1.1" 200 - 01/14/2026 02:12:20 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:23 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:28 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:30 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=k3-j721e-sk has no online devices 01/14/2026 02:12:33 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-axg-s400 has no online devices 01/14/2026 02:12:34 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-gxl-s905d-p230 has no online devices Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 01/14/2026 02:12:35 PM UTC [DEBUG] Job node created: 6967a45349c7a262599a93b1. Parent: 695fb0c7258e18ba90402535 127.0.0.1 - - [14/Jan/2026 14:12:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:51] "GET /health HTTP/1.1" 200 - 01/14/2026 02:21:52 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 603s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 02:21:52 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:21:52 PM UTC [ERROR] Thread: scheduler-retry, stuck for 603s 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 02:21:54 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:21:54 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:21:54 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:21:54 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:21:54 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:21:54 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:21:54 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:21:54 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:21:54 PM UTC [DEBUG] Node channel sub id: 102839 01/14/2026 02:21:54 PM UTC [DEBUG] Retry channel sub id: 102840 01/14/2026 02:21:54 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:21:54 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:21:54 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:21:54 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:22:22] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:22:52] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:23:22] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:23:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:24:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:24:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:25:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:25:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:26:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:26:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:27:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:27:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:28:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:28:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:29:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:29:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:30:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:30:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:31:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:31:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:32:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:32:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:33:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:33:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:34:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:34:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:35:24] "GET /health HTTP/1.1" 200 - 01/14/2026 02:35:26 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:35:28 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:35:33 PM UTC [DEBUG] Job node created: 6967a9b549c7a262599a93c1. Parent: 6967512dc8d7fe90f71efafd Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:35:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:36:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:36:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:37:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:37:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:38:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:38:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:39:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:39:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:40:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:40:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:41:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:41:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:42:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:42:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:43:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:43:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:44:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:44:56] "GET /health HTTP/1.1" 200 - 01/14/2026 02:45:24 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 620s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:45:24 PM UTC [ERROR] ================================================================================ 01/14/2026 02:45:24 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:45:24 PM UTC [ERROR] Thread: scheduler-retry, stuck for 620s 01/14/2026 02:45:24 PM UTC [ERROR] ================================================================================ 01/14/2026 01:58:18 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 01:58:19 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 01:58:19 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 01:58:19 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:20 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 01:58:20 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 01:58:20 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 01:58:20 PM UTC [DEBUG] Node channel sub id: 102826 01/14/2026 01:58:20 PM UTC [DEBUG] Retry channel sub id: 102827 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 13:58:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:11] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:42] "GET /health HTTP/1.1" 200 - 01/14/2026 02:01:00 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:03 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:04 PM UTC [DEBUG] Job node created: 6967a1a049c7a262599a9397. Parent: 695fb0b4258e18ba90402526 Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:01:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:01:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:43] "GET /health HTTP/1.1" 200 - 01/14/2026 02:10:50 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 616s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:50 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:10:50 PM UTC [ERROR] Thread: scheduler-retry, stuck for 616s 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:51 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:10:52 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:10:52 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:10:52 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:10:52 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:10:52 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:10:52 PM UTC [DEBUG] Node channel sub id: 102835 01/14/2026 02:10:52 PM UTC [DEBUG] Retry channel sub id: 102836 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:11:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:11:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:12:20] "GET /health HTTP/1.1" 200 - 01/14/2026 02:12:20 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:23 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:28 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:30 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=k3-j721e-sk has no online devices 01/14/2026 02:12:33 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-axg-s400 has no online devices 01/14/2026 02:12:34 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-gxl-s905d-p230 has no online devices Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 01/14/2026 02:12:35 PM UTC [DEBUG] Job node created: 6967a45349c7a262599a93b1. Parent: 695fb0c7258e18ba90402535 127.0.0.1 - - [14/Jan/2026 14:12:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:51] "GET /health HTTP/1.1" 200 - 01/14/2026 02:21:52 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 603s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 02:21:52 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:21:52 PM UTC [ERROR] Thread: scheduler-retry, stuck for 603s 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 02:21:54 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:21:54 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:21:54 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:21:54 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:21:54 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:21:54 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:21:54 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:21:54 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:21:54 PM UTC [DEBUG] Node channel sub id: 102839 01/14/2026 02:21:54 PM UTC [DEBUG] Retry channel sub id: 102840 01/14/2026 02:21:54 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:21:54 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:21:54 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:21:54 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:22:22] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:22:52] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:23:22] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:23:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:24:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:24:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:25:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:25:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:26:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:26:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:27:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:27:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:28:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:28:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:29:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:29:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:30:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:30:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:31:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:31:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:32:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:32:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:33:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:33:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:34:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:34:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:35:24] "GET /health HTTP/1.1" 200 - 01/14/2026 02:35:26 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:35:28 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:35:33 PM UTC [DEBUG] Job node created: 6967a9b549c7a262599a93c1. Parent: 6967512dc8d7fe90f71efafd Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:35:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:36:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:36:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:37:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:37:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:38:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:38:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:39:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:39:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:40:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:40:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:41:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:41:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:42:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:42:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:43:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:43:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:44:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:44:56] "GET /health HTTP/1.1" 200 - 01/14/2026 02:45:24 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 620s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:45:24 PM UTC [ERROR] ================================================================================ 01/14/2026 02:45:24 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:45:24 PM UTC [ERROR] Thread: scheduler-retry, stuck for 620s 01/14/2026 02:45:24 PM UTC [ERROR] ================================================================================ 01/14/2026 02:45:26 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:45:27 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:45:27 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:45:27 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:45:27 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:45:27 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:45:27 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:45:27 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:45:27 PM UTC [DEBUG] Node channel sub id: 102841 01/14/2026 02:45:27 PM UTC [DEBUG] Retry channel sub id: 102842 01/14/2026 02:45:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:45:27 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:45:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:45:27 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:45:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:46:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:46:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:47:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:47:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:48:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:48:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:49:25] "GET /health HTTP/1.1" 200 - 01/14/2026 02:49:27 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:49:30 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner 01/14/2026 02:49:32 PM UTC [DEBUG] Job node created: 6967acfc49c7a262599a93ed. Parent: 69675142c8d7fe90f71efb0c self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:49:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:50:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:50:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:51:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:51:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:52:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:52:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:53:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:53:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:54:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:54:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:55:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:55:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:56:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:56:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:57:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:57:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:58:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:58:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:59:27] "GET /health HTTP/1.1" 200 - 01/14/2026 02:59:27 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 608s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:59:27 PM UTC [ERROR] ================================================================================ 01/14/2026 02:59:27 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:59:27 PM UTC [ERROR] Thread: scheduler-retry, stuck for 608s 01/14/2026 02:59:27 PM UTC [ERROR] ================================================================================ 01/14/2026 01:58:18 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 01:58:19 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 01:58:19 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:19 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 01:58:19 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 01:58:20 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 01:58:20 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 01:58:20 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 01:58:20 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 01:58:20 PM UTC [DEBUG] Node channel sub id: 102826 01/14/2026 01:58:20 PM UTC [DEBUG] Retry channel sub id: 102827 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 01:58:20 PM UTC [INFO] Listening for available checkout events 01/14/2026 01:58:20 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 13:58:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:11] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 13:59:41] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:00:42] "GET /health HTTP/1.1" 200 - 01/14/2026 02:01:00 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:03 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:01:04 PM UTC [DEBUG] Job node created: 6967a1a049c7a262599a9397. Parent: 695fb0b4258e18ba90402526 Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:01:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:01:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:02:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:03:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:04:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:05:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:12] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:06:42] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:07:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:08:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:09:43] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:13] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:10:43] "GET /health HTTP/1.1" 200 - 01/14/2026 02:10:50 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 616s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:50 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:10:50 PM UTC [ERROR] Thread: scheduler-retry, stuck for 616s 01/14/2026 02:10:50 PM UTC [ERROR] ================================================================================ 01/14/2026 02:10:51 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:10:52 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:10:52 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:10:52 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:10:52 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:10:52 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:10:52 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:10:52 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:10:52 PM UTC [DEBUG] Node channel sub id: 102835 01/14/2026 02:10:52 PM UTC [DEBUG] Retry channel sub id: 102836 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:10:52 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:11:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:11:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:12:20] "GET /health HTTP/1.1" 200 - 01/14/2026 02:12:20 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:23 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:28 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:12:30 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=k3-j721e-sk has no online devices 01/14/2026 02:12:33 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-axg-s400 has no online devices 01/14/2026 02:12:34 PM UTC [INFO] Skipping job baseline-arm64-baylibre for lava-baylibre: device_type=meson-gxl-s905d-p230 has no online devices Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 01/14/2026 02:12:35 PM UTC [DEBUG] Job node created: 6967a45349c7a262599a93b1. Parent: 695fb0c7258e18ba90402535 127.0.0.1 - - [14/Jan/2026 14:12:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:13:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:14:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:15:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:16:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:17:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:18:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:19:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:20:51] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:21] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:21:51] "GET /health HTTP/1.1" 200 - 01/14/2026 02:21:52 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 603s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 02:21:52 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:21:52 PM UTC [ERROR] Thread: scheduler-retry, stuck for 603s 01/14/2026 02:21:52 PM UTC [ERROR] ================================================================================ 01/14/2026 02:21:54 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:21:54 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:21:54 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:21:54 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:21:54 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:21:54 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:21:54 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:21:54 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:21:54 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:21:54 PM UTC [DEBUG] Node channel sub id: 102839 01/14/2026 02:21:54 PM UTC [DEBUG] Retry channel sub id: 102840 01/14/2026 02:21:54 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:21:54 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:21:54 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:21:54 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:22:22] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:22:52] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:23:22] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:23:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:24:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:24:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:25:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:25:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:26:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:26:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:27:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:27:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:28:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:28:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:29:23] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:29:53] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:30:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:30:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:31:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:31:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:32:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:32:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:33:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:33:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:34:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:34:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:35:24] "GET /health HTTP/1.1" 200 - 01/14/2026 02:35:26 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:35:28 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm 01/14/2026 02:35:33 PM UTC [DEBUG] Job node created: 6967a9b549c7a262599a93c1. Parent: 6967512dc8d7fe90f71efafd Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:35:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:36:24] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:36:54] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:37:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:37:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:38:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:38:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:39:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:39:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:40:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:40:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:41:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:41:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:42:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:42:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:43:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:43:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:44:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:44:56] "GET /health HTTP/1.1" 200 - 01/14/2026 02:45:24 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 620s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:45:24 PM UTC [ERROR] ================================================================================ 01/14/2026 02:45:24 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:45:24 PM UTC [ERROR] Thread: scheduler-retry, stuck for 620s 01/14/2026 02:45:24 PM UTC [ERROR] ================================================================================ 01/14/2026 02:45:26 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:45:27 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:45:27 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:45:27 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:45:27 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:45:27 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:45:27 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:45:27 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:45:27 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:45:27 PM UTC [DEBUG] Node channel sub id: 102841 01/14/2026 02:45:27 PM UTC [DEBUG] Retry channel sub id: 102842 01/14/2026 02:45:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:45:27 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:45:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:45:27 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:45:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:46:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:46:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:47:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:47:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:48:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:48:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:49:25] "GET /health HTTP/1.1" 200 - 01/14/2026 02:49:27 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.ciplatform.org/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 01/14/2026 02:49:30 PM UTC [WARNING] Failed to check LAVA queue depth for qemu-arm64: 400 Client Error: Bad Request for url: https://lava.collabora.dev/api/v0.2/devices/?format=json&limit=256&device_type=qemu-arm64 Exception in thread scheduler-retry: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner 01/14/2026 02:49:32 PM UTC [DEBUG] Job node created: 6967acfc49c7a262599a93ed. Parent: 69675142c8d7fe90f71efb0c self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 14:49:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:50:25] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:50:55] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:51:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:51:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:52:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:52:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:53:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:53:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:54:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:54:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:55:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:55:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:56:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:56:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:57:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:57:56] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:58:26] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:58:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 14:59:27] "GET /health HTTP/1.1" 200 - 01/14/2026 02:59:27 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-retry' stuck for 608s (timeout: 600s). Forcing immediate exit! 01/14/2026 02:59:27 PM UTC [ERROR] ================================================================================ 01/14/2026 02:59:27 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 02:59:27 PM UTC [ERROR] Thread: scheduler-retry, stuck for 608s 01/14/2026 02:59:27 PM UTC [ERROR] ================================================================================ 01/14/2026 02:59:29 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 02:59:29 PM UTC [INFO] Runtimes from KContext: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:59:29 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 02:59:29 PM UTC [INFO] Initializing runtimes: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:59:29 PM UTC [INFO] Filtering runtimes by name: ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-qualcomm', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:59:29 PM UTC [WARNING] Runtime 'lava-qualcomm' not found in configuration 01/14/2026 02:59:29 PM UTC [INFO] Selected 11 runtime(s): ['lava-collabora', 'lava-collabora-staging', 'lava-clabbe', 'lava-broonie', 'lava-baylibre', 'lava-cip', 'lava-pengutronix', 'lava-kontron', 'lava-foundriesio', 'lava-testpull', 'pull-labs-demo'] 01/14/2026 02:59:29 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 02:59:29 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 02:59:29 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 02:59:29 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 02:59:29 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 02:59:29 PM UTC [DEBUG] Node channel sub id: 102843 01/14/2026 02:59:29 PM UTC [DEBUG] Retry channel sub id: 102844 01/14/2026 02:59:29 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:59:29 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 02:59:29 PM UTC [INFO] Listening for available checkout events 01/14/2026 02:59:29 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 14:59:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:00:27] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:00:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:01:27] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:01:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:02:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:02:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:03:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:03:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:04:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:04:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:05:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:05:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:06:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:06:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:07:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:07:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:08:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:08:59] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:09:29] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:09:59] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:10:29] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:10:59] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:11:29] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:11:59] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:12:29] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:12:59] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:13:29] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:13:59] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:14:29] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:14:59] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:15:29] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:16:00] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:16:30] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:17:00] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:17:30] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:18:00] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:18:30] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:19:00] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:19:30] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:20:00] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:20:30] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:21:00] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:21:30] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:22:00] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:22:30] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:23:01] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:23:31] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:24:01] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:24:31] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:25:01] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:25:31] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:26:01] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:26:31] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:27:01] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:27:31] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:28:01] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:28:31] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:29:01] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:29:31] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:30:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:30:32] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:31:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:31:32] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:32:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:32:32] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:33:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:33:32] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:34:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:34:32] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:35:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:35:32] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:36:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:36:32] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:37:02] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:37:33] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:38:03] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:38:33] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:39:03] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:39:33] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:40:03] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:40:33] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:41:03] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:41:33] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:42:03] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:42:33] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:43:03] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:43:33] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:44:03] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:44:34] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:45:04] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:45:34] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:46:04] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:46:34] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:47:04] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:47:34] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:48:04] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:48:34] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:49:04] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:49:34] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:50:04] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:50:34] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:51:04] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:51:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:52:05] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:52:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:53:05] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:53:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:54:05] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:54:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:55:05] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:55:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:56:05] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:56:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:57:05] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:57:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:58:05] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:58:35] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:59:06] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 15:59:36] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:00:06] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:00:36] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:01:06] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:01:36] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:02:06] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:02:36] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:03:06] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:03:36] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:04:06] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:04:36] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:05:06] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:05:37] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:06:07] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:06:37] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:07:07] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:07:37] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:08:07] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:08:37] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:09:07] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:09:37] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:10:07] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:10:37] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:11:07] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:11:37] "GET /health HTTP/1.1" 200 - 01/14/2026 04:11:48 PM UTC [INFO] Stopping. 01/14/2026 04:11:48 PM UTC [INFO] Unsubscribing: 102843 01/14/2026 04:11:48 PM UTC [INFO] Unsubscribing: 102844