08/06/2025 12:41:08 AM UTC [INFO] Looking for nodes with lapsed holdoff...
08/06/2025 12:41:08 AM UTC [INFO] Press Ctrl-C to stop.
/home/kernelci/pipeline/src/timeout.py:119: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
  'holdoff__lt': datetime.isoformat(datetime.utcnow()),
08/06/2025 12:41:08 AM UTC [DEBUG] 68929fbbbbc792f48746d08e RUNNING: 0
08/06/2025 12:41:08 AM UTC [DEBUG] 68929fbbbbc792f48746d08e RUNNING (recursive): 0
08/06/2025 12:41:08 AM UTC [DEBUG] 68929fbbbbc792f48746d08e DONE
08/06/2025 12:43:09 AM UTC [DEBUG] 68929c762d387824780d72a9 RUNNING: 0
08/06/2025 12:43:09 AM UTC [DEBUG] 68929c762d387824780d72a9 RUNNING (recursive): 0
08/06/2025 12:43:09 AM UTC [DEBUG] 68929f83bbc792f48746d05e RUNNING: 0
08/06/2025 12:43:09 AM UTC [DEBUG] 68929f83bbc792f48746d05e RUNNING (recursive): 0
08/06/2025 12:43:09 AM UTC [DEBUG] 68929f94bbc792f48746d06d RUNNING: 23
08/06/2025 12:43:09 AM UTC [DEBUG] 68929fa2bbc792f48746d079 RUNNING: 0
08/06/2025 12:43:09 AM UTC [DEBUG] 68929fa2bbc792f48746d079 RUNNING (recursive): 0
08/06/2025 12:43:09 AM UTC [DEBUG] 68929f94bbc792f48746d06d HOLDOFF
08/06/2025 12:43:09 AM UTC [DEBUG] 68929c762d387824780d72a9 DONE
08/06/2025 12:43:09 AM UTC [DEBUG] 68929f83bbc792f48746d05e DONE
08/06/2025 12:43:09 AM UTC [DEBUG] 68929fa2bbc792f48746d079 DONE
08/06/2025 12:44:09 AM UTC [DEBUG] 68929f9fbbc792f48746d076 RUNNING: 0
08/06/2025 12:44:09 AM UTC [DEBUG] 68929f9fbbc792f48746d076 RUNNING (recursive): 0
08/06/2025 12:44:09 AM UTC [DEBUG] 68929f9fbbc792f48746d076 DONE
08/06/2025 12:45:09 AM UTC [DEBUG] 68929f9bbbc792f48746d073 RUNNING: 2
08/06/2025 12:45:09 AM UTC [DEBUG] 68929f9bbbc792f48746d073 HOLDOFF
08/06/2025 12:47:09 AM UTC [DEBUG] 68929f98bbc792f48746d070 RUNNING: 0
08/06/2025 12:47:09 AM UTC [DEBUG] 68929f98bbc792f48746d070 RUNNING (recursive): 0
08/06/2025 12:47:09 AM UTC [DEBUG] 68929f98bbc792f48746d070 DONE
08/06/2025 12:49:09 AM UTC [DEBUG] 68929c7f2d387824780d72b2 RUNNING: 0
08/06/2025 12:49:09 AM UTC [DEBUG] 68929c7f2d387824780d72b2 RUNNING (recursive): 0
08/06/2025 12:49:09 AM UTC [DEBUG] 68929f91bbc792f48746d06a RUNNING: 0
08/06/2025 12:49:09 AM UTC [DEBUG] 68929f91bbc792f48746d06a RUNNING (recursive): 0
08/06/2025 12:49:09 AM UTC [DEBUG] 68929c7f2d387824780d72b2 DONE
08/06/2025 12:49:09 AM UTC [DEBUG] 68929f91bbc792f48746d06a DONE
08/06/2025 12:50:09 AM UTC [DEBUG] 68929c8a2d387824780d72bb RUNNING: 9
08/06/2025 12:50:09 AM UTC [DEBUG] 68929c8a2d387824780d72bb HOLDOFF
08/06/2025 12:53:10 AM UTC [DEBUG] 6892a4a822d3cc28955e61ed RUNNING: 21
08/06/2025 12:53:10 AM UTC [DEBUG] 6892a4a822d3cc28955e61ed HOLDOFF
08/06/2025 12:58:10 AM UTC [DEBUG] 68929f7fbbc792f48746d05b RUNNING: 0
08/06/2025 12:58:10 AM UTC [DEBUG] 68929f7fbbc792f48746d05b RUNNING (recursive): 0
08/06/2025 12:58:10 AM UTC [DEBUG] 68929f7fbbc792f48746d05b DONE
08/06/2025 01:00:10 AM UTC [DEBUG] 68929c8d2d387824780d72be RUNNING: 0
08/06/2025 01:00:10 AM UTC [DEBUG] 68929c8d2d387824780d72be RUNNING (recursive): 0
08/06/2025 01:00:10 AM UTC [DEBUG] 68929c952d387824780d72c4 RUNNING: 0
08/06/2025 01:00:10 AM UTC [DEBUG] 68929c952d387824780d72c4 RUNNING (recursive): 0
08/06/2025 01:00:10 AM UTC [DEBUG] 6892a52222d3cc28955e627d RUNNING: 0
08/06/2025 01:00:10 AM UTC [DEBUG] 6892a52222d3cc28955e627d RUNNING (recursive): 0
08/06/2025 01:00:10 AM UTC [DEBUG] 68929c8d2d387824780d72be DONE
08/06/2025 01:00:10 AM UTC [DEBUG] 68929c952d387824780d72c4 DONE
08/06/2025 01:00:10 AM UTC [DEBUG] 6892a52222d3cc28955e627d DONE
08/06/2025 01:01:10 AM UTC [DEBUG] 6892a4fc22d3cc28955e6244 RUNNING: 0
08/06/2025 01:01:10 AM UTC [DEBUG] 6892a4fc22d3cc28955e6244 RUNNING (recursive): 0
08/06/2025 01:01:10 AM UTC [DEBUG] 6892a4fc22d3cc28955e6244 DONE
08/06/2025 01:04:10 AM UTC [DEBUG] 6892a4f522d3cc28955e6236 RUNNING: 0
08/06/2025 01:04:10 AM UTC [DEBUG] 6892a4f522d3cc28955e6236 RUNNING (recursive): 0
08/06/2025 01:04:10 AM UTC [DEBUG] 6892a4f522d3cc28955e6236 DONE
08/06/2025 01:05:11 AM UTC [DEBUG] 6892a50622d3cc28955e624d RUNNING: 0
08/06/2025 01:05:11 AM UTC [DEBUG] 6892a50622d3cc28955e624d RUNNING (recursive): 0
08/06/2025 01:05:11 AM UTC [DEBUG] 6892a50622d3cc28955e624d DONE
08/06/2025 01:06:11 AM UTC [DEBUG] 68929f86bbc792f48746d061 RUNNING: 0
08/06/2025 01:06:11 AM UTC [DEBUG] 68929f86bbc792f48746d061 RUNNING (recursive): 0
08/06/2025 01:06:11 AM UTC [DEBUG] 68929f86bbc792f48746d061 DONE
08/06/2025 01:08:11 AM UTC [DEBUG] 68929f7ebbc792f48746d058 RUNNING: 0
08/06/2025 01:08:11 AM UTC [DEBUG] 68929f7ebbc792f48746d058 RUNNING (recursive): 0
08/06/2025 01:08:11 AM UTC [DEBUG] 68929fa9bbc792f48746d07f RUNNING: 0
08/06/2025 01:08:11 AM UTC [DEBUG] 68929fa9bbc792f48746d07f RUNNING (recursive): 0
08/06/2025 01:08:11 AM UTC [DEBUG] 6892a51f22d3cc28955e6276 RUNNING: 0
08/06/2025 01:08:11 AM UTC [DEBUG] 6892a51f22d3cc28955e6276 RUNNING (recursive): 0
08/06/2025 01:08:11 AM UTC [DEBUG] 68929f7ebbc792f48746d058 DONE
08/06/2025 01:08:11 AM UTC [DEBUG] 68929fa9bbc792f48746d07f DONE
08/06/2025 01:08:11 AM UTC [DEBUG] 6892a51f22d3cc28955e6276 DONE
08/06/2025 01:09:11 AM UTC [DEBUG] 6892a4eb22d3cc28955e6221 RUNNING: 0
08/06/2025 01:09:11 AM UTC [DEBUG] 6892a4eb22d3cc28955e6221 RUNNING (recursive): 0
08/06/2025 01:09:11 AM UTC [DEBUG] 6892a4ff22d3cc28955e6247 RUNNING: 1
08/06/2025 01:09:11 AM UTC [DEBUG] 6892a4ff22d3cc28955e6247 HOLDOFF
08/06/2025 01:09:11 AM UTC [DEBUG] 6892a4eb22d3cc28955e6221 DONE
08/06/2025 01:10:12 AM UTC [DEBUG] 68929c982d387824780d72c7 RUNNING: 0
08/06/2025 01:10:12 AM UTC [DEBUG] 68929c982d387824780d72c7 RUNNING (recursive): 0
08/06/2025 01:10:12 AM UTC [DEBUG] 68929fb7bbc792f48746d08b RUNNING: 1
08/06/2025 01:10:12 AM UTC [DEBUG] 68929fb7bbc792f48746d08b HOLDOFF
08/06/2025 01:10:12 AM UTC [DEBUG] 68929c982d387824780d72c7 DONE
08/06/2025 01:14:12 AM UTC [DEBUG] 6892a4f822d3cc28955e6239 RUNNING: 25
08/06/2025 01:14:12 AM UTC [DEBUG] 6892a51722d3cc28955e6268 RUNNING: 0
08/06/2025 01:14:12 AM UTC [DEBUG] 6892a51722d3cc28955e6268 RUNNING (recursive): 0
08/06/2025 01:14:12 AM UTC [DEBUG] 6892a4f822d3cc28955e6239 HOLDOFF
08/06/2025 01:14:12 AM UTC [DEBUG] 6892a51722d3cc28955e6268 DONE
08/06/2025 01:15:12 AM UTC [DEBUG] 6892a4e722d3cc28955e621a RUNNING: 0
08/06/2025 01:15:12 AM UTC [DEBUG] 6892a4e722d3cc28955e621a RUNNING (recursive): 0
08/06/2025 01:15:12 AM UTC [DEBUG] 6892a50322d3cc28955e624a RUNNING: 0
08/06/2025 01:15:12 AM UTC [DEBUG] 6892a50322d3cc28955e624a RUNNING (recursive): 0
08/06/2025 01:15:12 AM UTC [DEBUG] 6892a4e722d3cc28955e621a DONE
08/06/2025 01:15:12 AM UTC [DEBUG] 6892a50322d3cc28955e624a DONE
08/06/2025 01:17:12 AM UTC [DEBUG] 68929f89bbc792f48746d064 RUNNING: 0
08/06/2025 01:17:12 AM UTC [DEBUG] 68929f89bbc792f48746d064 RUNNING (recursive): 0
08/06/2025 01:17:12 AM UTC [DEBUG] 68929fb0bbc792f48746d085 RUNNING: 0
08/06/2025 01:17:12 AM UTC [DEBUG] 68929fb0bbc792f48746d085 RUNNING (recursive): 0
08/06/2025 01:17:12 AM UTC [DEBUG] 68929f89bbc792f48746d064 DONE
08/06/2025 01:17:12 AM UTC [DEBUG] 68929fb0bbc792f48746d085 DONE
08/06/2025 01:18:12 AM UTC [DEBUG] 68929f8dbbc792f48746d067 RUNNING: 0
08/06/2025 01:18:12 AM UTC [DEBUG] 68929f8dbbc792f48746d067 RUNNING (recursive): 0
08/06/2025 01:18:12 AM UTC [DEBUG] 6892a4e422d3cc28955e620b RUNNING: 0
08/06/2025 01:18:13 AM UTC [DEBUG] 6892a4e422d3cc28955e620b RUNNING (recursive): 0
08/06/2025 01:18:13 AM UTC [DEBUG] 68929f8dbbc792f48746d067 DONE
08/06/2025 01:18:13 AM UTC [DEBUG] 6892a4e422d3cc28955e620b DONE
08/06/2025 01:19:13 AM UTC [DEBUG] 68929c912d387824780d72c1 RUNNING: 0
08/06/2025 01:19:13 AM UTC [DEBUG] 68929c912d387824780d72c1 RUNNING (recursive): 0
08/06/2025 01:19:13 AM UTC [DEBUG] 68929c912d387824780d72c1 DONE
08/06/2025 01:20:13 AM UTC [DEBUG] 6892a51b22d3cc28955e626f RUNNING: 1
08/06/2025 01:20:13 AM UTC [DEBUG] 6892a51b22d3cc28955e626f HOLDOFF
08/06/2025 01:21:13 AM UTC [DEBUG] 68929fadbbc792f48746d082 RUNNING: 0
08/06/2025 01:21:13 AM UTC [DEBUG] 68929fadbbc792f48746d082 RUNNING (recursive): 0
08/06/2025 01:21:13 AM UTC [DEBUG] 68929fadbbc792f48746d082 DONE
08/06/2025 01:22:13 AM UTC [DEBUG] 6892ab7922d3cc28955e7150 RUNNING: 27
08/06/2025 01:22:13 AM UTC [DEBUG] 6892ab7922d3cc28955e7150 HOLDOFF
08/06/2025 01:23:13 AM UTC [DEBUG] 68929fb4bbc792f48746d088 RUNNING: 0
08/06/2025 01:23:13 AM UTC [DEBUG] 68929fb4bbc792f48746d088 RUNNING (recursive): 0
08/06/2025 01:23:13 AM UTC [DEBUG] 68929fb4bbc792f48746d088 DONE
08/06/2025 01:26:13 AM UTC [DEBUG] 6892a51422d3cc28955e6265 RUNNING: 0
08/06/2025 01:26:13 AM UTC [DEBUG] 6892a51422d3cc28955e6265 RUNNING (recursive): 0
08/06/2025 01:26:13 AM UTC [DEBUG] 6892a51422d3cc28955e6265 DONE
08/06/2025 01:27:13 AM UTC [DEBUG] 6892a4e222d3cc28955e6208 RUNNING: 0
08/06/2025 01:27:14 AM UTC [DEBUG] 6892a4e222d3cc28955e6208 RUNNING (recursive): 0
08/06/2025 01:27:14 AM UTC [DEBUG] 6892a4ee22d3cc28955e6228 RUNNING: 0
08/06/2025 01:27:14 AM UTC [DEBUG] 6892a4ee22d3cc28955e6228 RUNNING (recursive): 0
08/06/2025 01:27:14 AM UTC [DEBUG] 6892a4e222d3cc28955e6208 DONE
08/06/2025 01:27:14 AM UTC [DEBUG] 6892a4ee22d3cc28955e6228 DONE
08/06/2025 01:28:14 AM UTC [DEBUG] 6892abce22d3cc28955e716d RUNNING: 0
08/06/2025 01:28:14 AM UTC [DEBUG] 6892abce22d3cc28955e716d RUNNING (recursive): 0
08/06/2025 01:28:14 AM UTC [DEBUG] 6892abce22d3cc28955e716d DONE
08/06/2025 01:31:14 AM UTC [DEBUG] 6892abf622d3cc28955e7191 RUNNING: 0
08/06/2025 01:31:14 AM UTC [DEBUG] 6892abf622d3cc28955e7191 RUNNING (recursive): 0
08/06/2025 01:31:14 AM UTC [DEBUG] 6892abf622d3cc28955e7191 DONE
08/06/2025 01:32:14 AM UTC [DEBUG] 6892a50a22d3cc28955e6254 RUNNING: 10
08/06/2025 01:32:14 AM UTC [DEBUG] 6892abf022d3cc28955e718b RUNNING: 0
08/06/2025 01:32:14 AM UTC [DEBUG] 6892abf022d3cc28955e718b RUNNING (recursive): 0
08/06/2025 01:32:14 AM UTC [DEBUG] 6892a50a22d3cc28955e6254 HOLDOFF
08/06/2025 01:32:14 AM UTC [DEBUG] 6892abf022d3cc28955e718b DONE
08/06/2025 01:33:14 AM UTC [DEBUG] 6892a4f122d3cc28955e622b RUNNING: 0
08/06/2025 01:33:14 AM UTC [DEBUG] 6892a4f122d3cc28955e622b RUNNING (recursive): 0
08/06/2025 01:33:14 AM UTC [DEBUG] 6892a4f122d3cc28955e622b DONE
08/06/2025 01:34:14 AM UTC [DEBUG] 6892abd122d3cc28955e7170 RUNNING: 0
08/06/2025 01:34:14 AM UTC [DEBUG] 6892abd122d3cc28955e7170 RUNNING (recursive): 0
08/06/2025 01:34:14 AM UTC [DEBUG] 6892abd122d3cc28955e7170 DONE
08/06/2025 01:37:15 AM UTC [DEBUG] 6892a51122d3cc28955e625e RUNNING: 0
08/06/2025 01:37:15 AM UTC [DEBUG] 6892a51122d3cc28955e625e RUNNING (recursive): 0
08/06/2025 01:37:15 AM UTC [DEBUG] 6892a51122d3cc28955e625e DONE
08/06/2025 01:38:15 AM UTC [DEBUG] 6892ac0122d3cc28955e7197 RUNNING: 0
08/06/2025 01:38:15 AM UTC [DEBUG] 6892ac0122d3cc28955e7197 RUNNING (recursive): 0
08/06/2025 01:38:15 AM UTC [DEBUG] 6892ac0122d3cc28955e7197 DONE
08/06/2025 01:39:15 AM UTC [DEBUG] 6892abdd22d3cc28955e7179 RUNNING: 17
08/06/2025 01:39:15 AM UTC [DEBUG] 6892abf322d3cc28955e718e RUNNING: 0
08/06/2025 01:39:15 AM UTC [DEBUG] 6892abf322d3cc28955e718e RUNNING (recursive): 0
08/06/2025 01:39:15 AM UTC [DEBUG] 6892abdd22d3cc28955e7179 HOLDOFF
08/06/2025 01:39:15 AM UTC [DEBUG] 6892abf322d3cc28955e718e DONE
08/06/2025 01:40:15 AM UTC [DEBUG] 6892abca22d3cc28955e716a RUNNING: 0
08/06/2025 01:40:15 AM UTC [DEBUG] 6892abca22d3cc28955e716a RUNNING (recursive): 0
08/06/2025 01:40:15 AM UTC [DEBUG] 6892abfa22d3cc28955e7194 RUNNING: 0
08/06/2025 01:40:16 AM UTC [DEBUG] 6892abfa22d3cc28955e7194 RUNNING (recursive): 0
08/06/2025 01:40:16 AM UTC [DEBUG] 6892abca22d3cc28955e716a DONE
08/06/2025 01:40:16 AM UTC [DEBUG] 6892abfa22d3cc28955e7194 DONE
08/06/2025 01:42:16 AM UTC [DEBUG] 6892abc722d3cc28955e7167 RUNNING: 0
08/06/2025 01:42:16 AM UTC [DEBUG] 6892abc722d3cc28955e7167 RUNNING (recursive): 0
08/06/2025 01:42:16 AM UTC [DEBUG] 6892ac0c22d3cc28955e71a0 RUNNING: 1
08/06/2025 01:42:16 AM UTC [DEBUG] 6892ac0c22d3cc28955e71a0 HOLDOFF
08/06/2025 01:42:16 AM UTC [DEBUG] 6892abc722d3cc28955e7167 DONE
08/06/2025 01:43:16 AM UTC [DEBUG] 6892abd522d3cc28955e7173 RUNNING: 21
08/06/2025 01:43:16 AM UTC [DEBUG] 6892abe322d3cc28955e717f RUNNING: 0
08/06/2025 01:43:16 AM UTC [DEBUG] 6892abe322d3cc28955e717f RUNNING (recursive): 0
08/06/2025 01:43:16 AM UTC [DEBUG] 6892ac0522d3cc28955e719a RUNNING: 1
08/06/2025 01:43:16 AM UTC [DEBUG] 6892abd522d3cc28955e7173 HOLDOFF
08/06/2025 01:43:16 AM UTC [DEBUG] 6892ac0522d3cc28955e719a HOLDOFF
08/06/2025 01:43:16 AM UTC [DEBUG] 6892abe322d3cc28955e717f DONE
08/06/2025 01:45:16 AM UTC [DEBUG] 6892abbb22d3cc28955e715b RUNNING: 0
08/06/2025 01:45:16 AM UTC [DEBUG] 6892abbb22d3cc28955e715b RUNNING (recursive): 0
08/06/2025 01:45:16 AM UTC [DEBUG] 6892abbb22d3cc28955e715b DONE
08/06/2025 01:47:16 AM UTC [DEBUG] 6892abd922d3cc28955e7176 RUNNING: 0
08/06/2025 01:47:16 AM UTC [DEBUG] 6892abd922d3cc28955e7176 RUNNING (recursive): 0
08/06/2025 01:47:16 AM UTC [DEBUG] 6892abd922d3cc28955e7176 DONE
08/06/2025 01:52:17 AM UTC [DEBUG] 6892abe622d3cc28955e7182 RUNNING: 0
08/06/2025 01:52:17 AM UTC [DEBUG] 6892abe622d3cc28955e7182 RUNNING (recursive): 0
08/06/2025 01:52:17 AM UTC [DEBUG] 6892abe622d3cc28955e7182 DONE
08/06/2025 01:55:17 AM UTC [DEBUG] 6892abb922d3cc28955e7158 RUNNING: 0
08/06/2025 01:55:17 AM UTC [DEBUG] 6892abb922d3cc28955e7158 RUNNING (recursive): 0
08/06/2025 01:55:17 AM UTC [DEBUG] 6892abe022d3cc28955e717c RUNNING: 0
08/06/2025 01:55:17 AM UTC [DEBUG] 6892abe022d3cc28955e717c RUNNING (recursive): 0
08/06/2025 01:55:17 AM UTC [DEBUG] 6892abb922d3cc28955e7158 DONE
08/06/2025 01:55:17 AM UTC [DEBUG] 6892abe022d3cc28955e717c DONE
08/06/2025 01:56:17 AM UTC [DEBUG] 6892abb622d3cc28955e7155 RUNNING: 0
08/06/2025 01:56:17 AM UTC [DEBUG] 6892abb622d3cc28955e7155 RUNNING (recursive): 0
08/06/2025 01:56:17 AM UTC [DEBUG] 6892abb622d3cc28955e7155 DONE
08/06/2025 01:57:17 AM UTC [DEBUG] 6892abe922d3cc28955e7185 RUNNING: 0
08/06/2025 01:57:17 AM UTC [DEBUG] 6892abe922d3cc28955e7185 RUNNING (recursive): 0
08/06/2025 01:57:17 AM UTC [DEBUG] 6892ac0822d3cc28955e719d RUNNING: 0
08/06/2025 01:57:17 AM UTC [DEBUG] 6892ac0822d3cc28955e719d RUNNING (recursive): 0
08/06/2025 01:57:17 AM UTC [DEBUG] 6892abe922d3cc28955e7185 DONE
08/06/2025 01:57:17 AM UTC [DEBUG] 6892ac0822d3cc28955e719d DONE
08/06/2025 02:01:17 AM UTC [DEBUG] 6892abed22d3cc28955e7188 RUNNING: 0
08/06/2025 02:01:17 AM UTC [DEBUG] 6892abed22d3cc28955e7188 RUNNING (recursive): 0
08/06/2025 02:01:17 AM UTC [DEBUG] 6892abed22d3cc28955e7188 DONE
08/06/2025 02:06:18 AM UTC [DEBUG] 68929fa6bbc792f48746d07c RUNNING: 12
08/06/2025 02:06:18 AM UTC [DEBUG] 68929fa6bbc792f48746d07c HOLDOFF
08/06/2025 02:23:18 AM UTC [DEBUG] 6892abc122d3cc28955e7161 RUNNING: 0
08/06/2025 02:23:18 AM UTC [DEBUG] 6892abc122d3cc28955e7161 RUNNING (recursive): 0
08/06/2025 02:23:18 AM UTC [DEBUG] 6892abc122d3cc28955e7161 DONE
08/06/2025 04:02:36 AM UTC [INFO] Stopping.