01/14/2026 04:12:24 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 04:12:26 PM UTC [INFO] Runtimes from KContext: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:26 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 04:12:26 PM UTC [INFO] Initializing runtimes: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:26 PM UTC [INFO] Filtering runtimes by name: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:26 PM UTC [INFO] Selected 2 runtime(s): ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:27 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 04:12:27 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 04:12:27 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 04:12:27 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 04:12:27 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 04:12:27 PM UTC [DEBUG] Node channel sub id: 102853 01/14/2026 04:12:27 PM UTC [DEBUG] Retry channel sub id: 102854 01/14/2026 04:12:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 04:12:27 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 04:12:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 04:12:27 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 16:12:48] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:13:18] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:13:48] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:14:18] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:14:48] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:15:18] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:15:48] "GET /health HTTP/1.1" 200 - rules[arch]: Arch x86_64 not allowed due ![] rules[arch]: Arch x86_64 not allowed due ![] rules[fragments]: Fragments missing one of ['coverage'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] 01/14/2026 04:16:10 PM UTC [DEBUG] Job node created: 6967c14a8779f4cfe2bcb903. Parent: 6967c0fb8779f4cfe2bcb8fc Exception in thread scheduler-node: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 16:16:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:16:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:17:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:17:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:18:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:18:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:19:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:19:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:20:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:20:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:21:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:21:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:22:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:22:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:23:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:23:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:24:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:24:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:25:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:25:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:26:20] "GET /health HTTP/1.1" 200 - 01/14/2026 04:26:27 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-node' stuck for 622s (timeout: 600s). Forcing immediate exit! 01/14/2026 04:26:27 PM UTC [ERROR] ================================================================================ 01/14/2026 04:26:27 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 04:26:27 PM UTC [ERROR] Thread: scheduler-node, stuck for 622s 01/14/2026 04:26:27 PM UTC [ERROR] ================================================================================ 01/14/2026 04:12:24 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 04:12:26 PM UTC [INFO] Runtimes from KContext: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:26 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 04:12:26 PM UTC [INFO] Initializing runtimes: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:26 PM UTC [INFO] Filtering runtimes by name: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:26 PM UTC [INFO] Selected 2 runtime(s): ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:12:27 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 04:12:27 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 04:12:27 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 04:12:27 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 04:12:27 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 04:12:27 PM UTC [DEBUG] Node channel sub id: 102853 01/14/2026 04:12:27 PM UTC [DEBUG] Retry channel sub id: 102854 01/14/2026 04:12:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 04:12:27 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 04:12:27 PM UTC [INFO] Listening for available checkout events 01/14/2026 04:12:27 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 16:12:48] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:13:18] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:13:48] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:14:18] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:14:48] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:15:18] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:15:48] "GET /health HTTP/1.1" 200 - rules[arch]: Arch x86_64 not allowed due ![] rules[arch]: Arch x86_64 not allowed due ![] rules[fragments]: Fragments missing one of ['coverage'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] rules[tree]: Tree kernelci not allowed due ['chromiumos'] 01/14/2026 04:16:10 PM UTC [DEBUG] Job node created: 6967c14a8779f4cfe2bcb903. Parent: 6967c0fb8779f4cfe2bcb8fc Exception in thread scheduler-node: Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/home/kernelci/pipeline/./src/scheduler.py", line 768, in _run_scheduler self._run_job(job, runtime, platform, input_node, retry_counter) File "/home/kernelci/pipeline/./src/scheduler.py", line 460, in _run_job tree_priority = self._get_tree_priority(tree_name, branch_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kernelci/pipeline/./src/scheduler.py", line 425, in _get_tree_priority priority = build_config.priority ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BuildConfig' object has no attribute 'priority' 127.0.0.1 - - [14/Jan/2026 16:16:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:16:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:17:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:17:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:18:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:18:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:19:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:19:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:20:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:20:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:21:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:21:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:22:19] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:22:49] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:23:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:23:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:24:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:24:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:25:20] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:25:50] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:26:20] "GET /health HTTP/1.1" 200 - 01/14/2026 04:26:27 PM UTC [ERROR] WATCHDOG: Thread 'scheduler-node' stuck for 622s (timeout: 600s). Forcing immediate exit! 01/14/2026 04:26:27 PM UTC [ERROR] ================================================================================ 01/14/2026 04:26:27 PM UTC [ERROR] STUCK THREAD DETECTED - FORCING EXIT 01/14/2026 04:26:27 PM UTC [ERROR] Thread: scheduler-node, stuck for 622s 01/14/2026 04:26:27 PM UTC [ERROR] ================================================================================ 01/14/2026 04:26:28 PM UTC [INFO] Job backup disabled (set BACKUP_FILE_LIFETIME env var to enable) 01/14/2026 04:26:29 PM UTC [INFO] Runtimes from KContext: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:26:29 PM UTC [INFO] Runtime types from KContext: [] 01/14/2026 04:26:29 PM UTC [INFO] Initializing runtimes: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:26:29 PM UTC [INFO] Filtering runtimes by name: ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:26:29 PM UTC [INFO] Selected 2 runtime(s): ['k8s-gke-eu-west4', 'k8s-all'] 01/14/2026 04:26:30 PM UTC [INFO] Default storage config from KContext: kci-storage 01/14/2026 04:26:30 PM UTC [INFO] Attempting to initialize storage config: kci-storage 01/14/2026 04:26:30 PM UTC [INFO] KContext get_storage_config returned: True 01/14/2026 04:26:30 PM UTC [INFO] KContext storage initialization successful: True 01/14/2026 04:26:30 PM UTC [INFO] Watchdog started with 600s timeout 01/14/2026 04:26:30 PM UTC [DEBUG] Node channel sub id: 102857 01/14/2026 04:26:30 PM UTC [DEBUG] Retry channel sub id: 102858 01/14/2026 04:26:30 PM UTC [INFO] Listening for available checkout events 01/14/2026 04:26:30 PM UTC [INFO] Press Ctrl-C to stop. 01/14/2026 04:26:30 PM UTC [INFO] Listening for available checkout events 01/14/2026 04:26:30 PM UTC [INFO] Press Ctrl-C to stop. 127.0.0.1 - - [14/Jan/2026 16:26:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:27:27] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:27:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:28:27] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:28:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:29:27] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:29:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:30:27] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:30:57] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:31:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:31:58] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:32:28] "GET /health HTTP/1.1" 200 - 127.0.0.1 - - [14/Jan/2026 16:32:58] "GET /health HTTP/1.1" 200 - 01/14/2026 04:32:59 PM UTC [INFO] Stopping. 01/14/2026 04:32:59 PM UTC [INFO] Unsubscribing: 102857 01/14/2026 04:32:59 PM UTC [INFO] Unsubscribing: 102858