INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.19.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54400 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.19.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35976 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:31:30,384 - lava_callback - INFO - Received callback for node 69844c20969566dcb0d2d14e 2026-02-06 00:31:30,393 - lava_callback - INFO - Starting async job submit for node 69844c20969566dcb0d2d14e INFO: 172.19.0.1:34872 - "POST /node/69844c20969566dcb0d2d14e HTTP/1.0" 202 Accepted 2026-02-06 00:31:30,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:31:30,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:30,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:31:30,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:30,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:30,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:31:32,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:32,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:32,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:31:36,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:36,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:36,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:34884 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:31:44,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:44,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:44,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60610 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:32:00,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:32:00,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:32:00,461 - lava_callback - ERROR - Error processing callback for node 69844c20969566dcb0d2d14e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69844c20969566dcb0d2d14e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69844c20969566dcb0d2d14e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69844c20969566dcb0d2d14e (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49518 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:41:58,396 - lava_callback - INFO - Received callback for node 698295c93507dcddca58e1dc INFO: 172.19.0.1:49534 - "POST /node/698295c93507dcddca58e1dc HTTP/1.0" 202 Accepted 2026-02-06 00:41:58,403 - lava_callback - INFO - Starting async job submit for node 698295c93507dcddca58e1dc 2026-02-06 00:41:58,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:41:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:41:58,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:58,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:41:58,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:41:58,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,059 - lava_callback - INFO - Received callback for node 698295c93507dcddca58e1de 2026-02-06 00:41:59,064 - lava_callback - INFO - Starting async job submit for node 698295c93507dcddca58e1de INFO: 172.19.0.1:49548 - "POST /node/698295c93507dcddca58e1de HTTP/1.0" 202 Accepted 2026-02-06 00:41:59,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:41:59,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:41:59,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:41:59,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:41:59,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,727 - lava_callback - INFO - Received callback for node 698295c93507dcddca58e1e0 2026-02-06 00:41:59,732 - lava_callback - INFO - Starting async job submit for node 698295c93507dcddca58e1e0 INFO: 172.19.0.1:56020 - "POST /node/698295c93507dcddca58e1e0 HTTP/1.0" 202 Accepted 2026-02-06 00:41:59,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:41:59,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:41:59,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:41:59,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:41:59,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:00,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:00,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:00,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:01,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:01,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:01,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:01,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:01,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:01,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:04,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:04,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:04,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:05,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:05,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:05,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:05,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:05,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:05,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:56034 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:12,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:12,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:12,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:13,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:13,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:13,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:13,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:13,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:13,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:48380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59752 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:28,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:28,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:28,425 - lava_callback - ERROR - Error processing callback for node 698295c93507dcddca58e1dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1dc (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:42:29,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:29,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:29,087 - lava_callback - ERROR - Error processing callback for node 698295c93507dcddca58e1de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1de (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:42:29,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:29,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:29,754 - lava_callback - ERROR - Error processing callback for node 698295c93507dcddca58e1e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1e0 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:38360 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:38,300 - lava_callback - INFO - Received callback for node 698295f93507dcddca58e1eb INFO: 172.19.0.1:38374 - "POST /node/698295f93507dcddca58e1eb HTTP/1.0" 202 Accepted 2026-02-06 00:42:38,306 - lava_callback - INFO - Starting async job submit for node 698295f93507dcddca58e1eb 2026-02-06 00:42:38,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:42:38,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:38,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:38,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:38,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:38,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:38,986 - lava_callback - INFO - Received callback for node 698295fa3507dcddca58e1ed INFO: 172.19.0.1:38386 - "POST /node/698295fa3507dcddca58e1ed HTTP/1.0" 202 Accepted 2026-02-06 00:42:38,992 - lava_callback - INFO - Starting async job submit for node 698295fa3507dcddca58e1ed 2026-02-06 00:42:38,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:42:39,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:39,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:39,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:39,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:39,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:39,631 - lava_callback - INFO - Received callback for node 698295fa3507dcddca58e1ef INFO: 172.19.0.1:38400 - "POST /node/698295fa3507dcddca58e1ef HTTP/1.0" 202 Accepted 2026-02-06 00:42:39,637 - lava_callback - INFO - Starting async job submit for node 698295fa3507dcddca58e1ef 2026-02-06 00:42:39,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:42:39,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:39,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:39,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:39,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:39,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:40,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:40,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:40,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:41,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:41,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:41,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:41,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:41,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:41,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:44,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:44,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:44,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:45,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:45,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:45,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:45,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:45,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:45,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:60816 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:52,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:52,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:52,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:53,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:53,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:53,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:53,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:53,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:53,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42864 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:08,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:43:08,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:43:08,339 - lava_callback - ERROR - Error processing callback for node 698295f93507dcddca58e1eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295f93507dcddca58e1eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295f93507dcddca58e1eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295f93507dcddca58e1eb (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:43:09,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:43:09,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:43:09,054 - lava_callback - ERROR - Error processing callback for node 698295fa3507dcddca58e1ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ed (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:43:09,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:43:09,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:43:09,658 - lava_callback - ERROR - Error processing callback for node 698295fa3507dcddca58e1ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ef (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:42514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42518 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:18,351 - lava_callback - INFO - Received callback for node 698296243507dcddca58e1f6 INFO: 172.19.0.1:42530 - "POST /node/698296243507dcddca58e1f6 HTTP/1.0" 202 Accepted 2026-02-06 00:43:18,357 - lava_callback - INFO - Starting async job submit for node 698296243507dcddca58e1f6 2026-02-06 00:43:18,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:43:18,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:18,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:18,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:18,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:18,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:20,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:20,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:20,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:24,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:24,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:24,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:37570 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:32,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:32,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:32,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46462 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:38,326 - lava_callback - INFO - Received callback for node 698296253507dcddca58e1f8 INFO: 172.19.0.1:46472 - "POST /node/698296253507dcddca58e1f8 HTTP/1.0" 202 Accepted 2026-02-06 00:43:38,332 - lava_callback - INFO - Starting async job submit for node 698296253507dcddca58e1f8 2026-02-06 00:43:38,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:43:38,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:38,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:38,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:38,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:38,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:39,012 - lava_callback - INFO - Received callback for node 698296253507dcddca58e1fa INFO: 172.19.0.1:46480 - "POST /node/698296253507dcddca58e1fa HTTP/1.0" 202 Accepted 2026-02-06 00:43:39,018 - lava_callback - INFO - Starting async job submit for node 698296253507dcddca58e1fa 2026-02-06 00:43:39,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:43:39,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:39,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:39,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:39,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:39,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:40,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:40,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:40,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:41,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:41,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:41,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:44,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:44,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:44,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:45,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:45,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:45,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:49112 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:48,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:48,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:48,378 - lava_callback - ERROR - Error processing callback for node 698296243507dcddca58e1f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296243507dcddca58e1f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296243507dcddca58e1f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296243507dcddca58e1f6 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:43:52,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:52,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:52,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:53,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:53,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:53,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51244 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:08,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:44:08,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:44:08,352 - lava_callback - ERROR - Error processing callback for node 698296253507dcddca58e1f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1f8 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:44:09,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:44:09,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:44:09,037 - lava_callback - ERROR - Error processing callback for node 698296253507dcddca58e1fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1fa (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:43800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43832 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:18,372 - lava_callback - INFO - Received callback for node 698296503507dcddca58e202 INFO: 172.19.0.1:43836 - "POST /node/698296503507dcddca58e202 HTTP/1.0" 202 Accepted 2026-02-06 00:44:18,378 - lava_callback - INFO - Starting async job submit for node 698296503507dcddca58e202 2026-02-06 00:44:18,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:18,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:18,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:18,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:18,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:18,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,272 - lava_callback - INFO - Received callback for node 698296503507dcddca58e204 INFO: 172.19.0.1:43850 - "POST /node/698296503507dcddca58e204 HTTP/1.0" 202 Accepted 2026-02-06 00:44:19,278 - lava_callback - INFO - Starting async job submit for node 698296503507dcddca58e204 2026-02-06 00:44:19,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:19,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:19,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:19,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:19,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,917 - lava_callback - INFO - Received callback for node 698296503507dcddca58e206 INFO: 172.19.0.1:55108 - "POST /node/698296503507dcddca58e206 HTTP/1.0" 202 Accepted 2026-02-06 00:44:19,923 - lava_callback - INFO - Starting async job submit for node 698296503507dcddca58e206 2026-02-06 00:44:19,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:19,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:19,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:19,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:20,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:20,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:20,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:21,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:21,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:21,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:21,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:21,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:21,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:24,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:24,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:24,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:25,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:25,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:25,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:25,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:25,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:25,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:55112 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:32,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:32,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:32,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:33,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:33,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:33,324 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:33,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:33,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40002 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:48,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:48,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:48,399 - lava_callback - ERROR - Error processing callback for node 698296503507dcddca58e202: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e202 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e202 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e202 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:44:49,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:49,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:49,326 - lava_callback - ERROR - Error processing callback for node 698296503507dcddca58e204: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e204 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e204 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e204 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:44:49,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:49,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:49,950 - lava_callback - ERROR - Error processing callback for node 698296503507dcddca58e206: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e206 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e206 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e206 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:59658 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:58,411 - lava_callback - INFO - Received callback for node 6982967f3507dcddca58e20f INFO: 172.19.0.1:59666 - "POST /node/6982967f3507dcddca58e20f HTTP/1.0" 202 Accepted 2026-02-06 00:44:58,416 - lava_callback - INFO - Starting async job submit for node 6982967f3507dcddca58e20f 2026-02-06 00:44:58,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:58,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:44:58,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:58,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:44:58,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:44:58,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,055 - lava_callback - INFO - Received callback for node 698296803507dcddca58e211 INFO: 172.19.0.1:59670 - "POST /node/698296803507dcddca58e211 HTTP/1.0" 202 Accepted 2026-02-06 00:44:59,060 - lava_callback - INFO - Starting async job submit for node 698296803507dcddca58e211 2026-02-06 00:44:59,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:59,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:44:59,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:44:59,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:44:59,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,698 - lava_callback - INFO - Received callback for node 698296803507dcddca58e213 INFO: 172.19.0.1:45802 - "POST /node/698296803507dcddca58e213 HTTP/1.0" 202 Accepted 2026-02-06 00:44:59,704 - lava_callback - INFO - Starting async job submit for node 698296803507dcddca58e213 2026-02-06 00:44:59,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:59,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:44:59,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:44:59,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:44:59,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:00,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:00,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:00,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:01,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:01,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:01,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:01,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:01,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:01,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:04,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:04,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:04,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:05,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:05,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:05,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:05,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:05,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:05,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:45818 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:45:12,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:12,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:12,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:13,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:13,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:13,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:13,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:13,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:13,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:48642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36126 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:45:28,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:28,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:28,459 - lava_callback - ERROR - Error processing callback for node 6982967f3507dcddca58e20f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6982967f3507dcddca58e20f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6982967f3507dcddca58e20f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6982967f3507dcddca58e20f (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:45:29,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:29,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:29,103 - lava_callback - ERROR - Error processing callback for node 698296803507dcddca58e211: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e211 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e211 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e211 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:45:29,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:29,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:29,723 - lava_callback - ERROR - Error processing callback for node 698296803507dcddca58e213: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e213 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e213 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e213 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48686 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:45:58,693 - lava_callback - INFO - Received callback for node 698296b13507dcddca58e27e INFO: 172.19.0.1:48696 - "POST /node/698296b13507dcddca58e27e HTTP/1.0" 202 Accepted 2026-02-06 00:45:58,698 - lava_callback - INFO - Starting async job submit for node 698296b13507dcddca58e27e 2026-02-06 00:45:58,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:45:58,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:45:58,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:58,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:45:58,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:45:58,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:59,352 - lava_callback - INFO - Received callback for node 698296b23507dcddca58e280 INFO: 172.19.0.1:48708 - "POST /node/698296b23507dcddca58e280 HTTP/1.0" 202 Accepted 2026-02-06 00:45:59,359 - lava_callback - INFO - Starting async job submit for node 698296b23507dcddca58e280 2026-02-06 00:45:59,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:45:59,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:45:59,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:59,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:45:59,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:45:59,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:59,996 - lava_callback - INFO - Received callback for node 698296b23507dcddca58e282 INFO: 172.19.0.1:48624 - "POST /node/698296b23507dcddca58e282 HTTP/1.0" 202 Accepted 2026-02-06 00:46:00,006 - lava_callback - INFO - Starting async job submit for node 698296b23507dcddca58e282 2026-02-06 00:46:00,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:46:00,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:00,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:00,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:00,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:00,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:00,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:00,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:00,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:01,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:01,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:01,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:02,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:02,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:02,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:04,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:04,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:04,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:05,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:05,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:05,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:06,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:06,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:06,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:48628 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:46:12,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:12,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:12,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:13,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:13,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:13,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:14,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:14,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:14,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50850 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:46:28,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:28,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:28,737 - lava_callback - ERROR - Error processing callback for node 698296b13507dcddca58e27e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b13507dcddca58e27e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b13507dcddca58e27e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b13507dcddca58e27e (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:46:29,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:29,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:29,408 - lava_callback - ERROR - Error processing callback for node 698296b23507dcddca58e280: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e280 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e280 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e280 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:46:30,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:30,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:30,029 - lava_callback - ERROR - Error processing callback for node 698296b23507dcddca58e282: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e282 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e282 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e282 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49106 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:52:11,174 - lava_callback - INFO - Received callback for node 698455b73a8748f28208488e INFO: 172.19.0.1:54282 - "POST /node/698455b73a8748f28208488e HTTP/1.0" 202 Accepted 2026-02-06 00:52:11,182 - lava_callback - INFO - Starting async job submit for node 698455b73a8748f28208488e 2026-02-06 00:52:11,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:52:11,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:11,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:52:11,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:11,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:11,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:52:13,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:13,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:13,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:54284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54314 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:52:17,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:17,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:17,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:52:25,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:25,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:25,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50612 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:52:41,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:41,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:41,245 - lava_callback - ERROR - Error processing callback for node 698455b73a8748f28208488e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b73a8748f28208488e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b73a8748f28208488e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b73a8748f28208488e (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43412 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:12:55,866 - lava_callback - INFO - Received callback for node 698455b83a8748f282084891 INFO: 172.19.0.1:47382 - "POST /node/698455b83a8748f282084891 HTTP/1.0" 202 Accepted 2026-02-06 01:12:55,875 - lava_callback - INFO - Starting async job submit for node 698455b83a8748f282084891 2026-02-06 01:12:55,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 01:12:55,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:12:55,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:12:55,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:12:55,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:12:55,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:47390 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:12:57,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:12:57,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:12:57,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:13:01,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:13:01,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:13:01,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:47514 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:13:09,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:13:09,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:13:09,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53244 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:13:25,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:13:25,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:13:25,951 - lava_callback - ERROR - Error processing callback for node 698455b83a8748f282084891: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b83a8748f282084891 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b83a8748f282084891 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b83a8748f282084891 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60128 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:54:19,800 - lava_callback - INFO - Received callback for node 698455b93a8748f282084894 INFO: 172.19.0.1:34902 - "POST /node/698455b93a8748f282084894 HTTP/1.0" 202 Accepted 2026-02-06 01:54:19,808 - lava_callback - INFO - Starting async job submit for node 698455b93a8748f282084894 2026-02-06 01:54:19,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 01:54:19,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:19,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:54:19,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:19,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:19,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:54:21,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:21,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:21,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:54:25,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:25,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:25,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:34908 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:54:33,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:33,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:33,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:32856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59290 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:54:49,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:49,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:49,900 - lava_callback - ERROR - Error processing callback for node 698455b93a8748f282084894: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b93a8748f282084894 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b93a8748f282084894 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b93a8748f282084894 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:34080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59026 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:14:49,160 - lava_callback - INFO - Received callback for node 698456903a8748f2820848f5 INFO: 172.19.0.1:59038 - "POST /node/698456903a8748f2820848f5 HTTP/1.0" 202 Accepted 2026-02-06 02:14:49,168 - lava_callback - INFO - Starting async job submit for node 698456903a8748f2820848f5 2026-02-06 02:14:49,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 02:14:49,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:49,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:14:49,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:14:49,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:49,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:14:51,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:14:51,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:51,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:14:55,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:14:55,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:55,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:42582 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:15:03,235 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:15:03,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:15:03,237 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38970 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:15:19,237 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:15:19,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:15:19,238 - lava_callback - ERROR - Error processing callback for node 698456903a8748f2820848f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456903a8748f2820848f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456903a8748f2820848f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456903a8748f2820848f5 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37406 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:35:42,066 - lava_callback - INFO - Received callback for node 698456913a8748f2820848f8 INFO: 172.19.0.1:46714 - "POST /node/698456913a8748f2820848f8 HTTP/1.0" 202 Accepted 2026-02-06 02:35:42,079 - lava_callback - INFO - Starting async job submit for node 698456913a8748f2820848f8 2026-02-06 02:35:42,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 02:35:42,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:42,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:35:42,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:42,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:42,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:35:44,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:44,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:44,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46726 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:35:48,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:48,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:48,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:35:56,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:56,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:56,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52084 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:36:12,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:36:12,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:36:12,221 - lava_callback - ERROR - Error processing callback for node 698456913a8748f2820848f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456913a8748f2820848f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456913a8748f2820848f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456913a8748f2820848f8 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:53214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37020 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:22,931 - lava_callback - INFO - Received callback for node 698456953a8748f2820848fb INFO: 172.19.0.1:36956 - "POST /node/698456953a8748f2820848fb HTTP/1.0" 202 Accepted 2026-02-06 02:56:22,938 - lava_callback - INFO - Starting async job submit for node 698456953a8748f2820848fb 2026-02-06 02:56:22,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 02:56:22,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:22,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:56:22,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:22,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:22,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:56:24,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:24,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:24,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:36964 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:28,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:28,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:28,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:33416 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:36,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:36,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:36,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:51256 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:53,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:53,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:53,001 - lava_callback - ERROR - Error processing callback for node 698456953a8748f2820848fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456953a8748f2820848fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456953a8748f2820848fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456953a8748f2820848fb (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:41052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48434 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:02,574 - lava_callback - INFO - Received callback for node 698456fc3a8748f2820849a6 INFO: 172.19.0.1:41200 - "POST /node/698456fc3a8748f2820849a6 HTTP/1.0" 202 Accepted 2026-02-06 03:17:02,582 - lava_callback - INFO - Starting async job submit for node 698456fc3a8748f2820849a6 2026-02-06 03:17:02,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 03:17:02,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:02,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:17:02,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:02,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:02,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:17:04,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:04,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:04,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:41212 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:08,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:08,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53450 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:16,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:16,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:16,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54324 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:32,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:32,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:32,649 - lava_callback - ERROR - Error processing callback for node 698456fc3a8748f2820849a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456fc3a8748f2820849a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456fc3a8748f2820849a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456fc3a8748f2820849a6 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33080 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:37:45,174 - lava_callback - INFO - Received callback for node 698456ff3a8748f2820849c7 INFO: 172.19.0.1:57786 - "POST /node/698456ff3a8748f2820849c7 HTTP/1.0" 202 Accepted 2026-02-06 03:37:45,182 - lava_callback - INFO - Starting async job submit for node 698456ff3a8748f2820849c7 2026-02-06 03:37:45,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 03:37:45,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:45,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:37:45,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:45,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:45,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:57800 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:37:47,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:47,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:47,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:37:51,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:51,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:51,253 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:41428 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:37:59,253 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:59,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:59,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57156 - "GET /api/metrics HTTP/1.0" 200 OK 2026-02-06 03:38:15,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:38:15,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:38:15,256 - lava_callback - ERROR - Error processing callback for node 698456ff3a8748f2820849c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456ff3a8748f2820849c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456ff3a8748f2820849c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456ff3a8748f2820849c7 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38064 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:16,830 - lava_callback - INFO - Received callback for node 698457033a8748f2820849fd INFO: 172.19.0.1:38076 - "POST /node/698457033a8748f2820849fd HTTP/1.0" 202 Accepted 2026-02-06 03:58:16,838 - lava_callback - INFO - Starting async job submit for node 698457033a8748f2820849fd 2026-02-06 03:58:16,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 03:58:16,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:16,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:58:16,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:16,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:16,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:58:18,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:18,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:18,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:38082 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:22,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:22,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:22,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:36396 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:30,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:30,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:30,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43496 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:46,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:46,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:46,907 - lava_callback - ERROR - Error processing callback for node 698457033a8748f2820849fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698457033a8748f2820849fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698457033a8748f2820849fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698457033a8748f2820849fd (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:43376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44116 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:10,546 - lava_callback - INFO - Received callback for node 698458523a8748f282084e0e INFO: 172.19.0.1:34724 - "POST /node/698458523a8748f282084e0e HTTP/1.0" 202 Accepted 2026-02-06 04:19:10,553 - lava_callback - INFO - Starting async job submit for node 698458523a8748f282084e0e 2026-02-06 04:19:10,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 04:19:10,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:10,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:19:10,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:10,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:10,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:19:12,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:12,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:12,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:34726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34742 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:16,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:16,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:16,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:39694 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:24,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:24,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:24,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35864 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:40,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:40,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:40,644 - lava_callback - ERROR - Error processing callback for node 698458523a8748f282084e0e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458523a8748f282084e0e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458523a8748f282084e0e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458523a8748f282084e0e (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35892 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:03,055 - lava_callback - INFO - Received callback for node 698458603a8748f282084e4a INFO: 172.19.0.1:46770 - "POST /node/698458603a8748f282084e4a HTTP/1.0" 202 Accepted 2026-02-06 04:40:03,063 - lava_callback - INFO - Starting async job submit for node 698458603a8748f282084e4a 2026-02-06 04:40:03,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 04:40:03,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:03,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:40:03,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:03,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:03,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:40:05,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:05,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:05,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46786 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:09,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:09,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:09,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53090 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:17,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:17,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:17,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49334 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:33,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:33,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:33,135 - lava_callback - ERROR - Error processing callback for node 698458603a8748f282084e4a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458603a8748f282084e4a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458603a8748f282084e4a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458603a8748f282084e4a (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1] INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.19.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54400 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.19.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35976 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:31:30,384 - lava_callback - INFO - Received callback for node 69844c20969566dcb0d2d14e 2026-02-06 00:31:30,393 - lava_callback - INFO - Starting async job submit for node 69844c20969566dcb0d2d14e INFO: 172.19.0.1:34872 - "POST /node/69844c20969566dcb0d2d14e HTTP/1.0" 202 Accepted 2026-02-06 00:31:30,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:31:30,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:30,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:31:30,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:30,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:30,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:31:32,455 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:32,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:32,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:31:36,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:36,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:36,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:34884 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:31:44,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:31:44,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:31:44,459 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69844c20969566dcb0d2d14e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60610 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:32:00,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69844c20969566dcb0d2d14e 2026-02-06 00:32:00,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69844c20969566dcb0d2d14e HTTP/1.1" 502 150 2026-02-06 00:32:00,461 - lava_callback - ERROR - Error processing callback for node 69844c20969566dcb0d2d14e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69844c20969566dcb0d2d14e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69844c20969566dcb0d2d14e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69844c20969566dcb0d2d14e (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49518 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:41:58,396 - lava_callback - INFO - Received callback for node 698295c93507dcddca58e1dc INFO: 172.19.0.1:49534 - "POST /node/698295c93507dcddca58e1dc HTTP/1.0" 202 Accepted 2026-02-06 00:41:58,403 - lava_callback - INFO - Starting async job submit for node 698295c93507dcddca58e1dc 2026-02-06 00:41:58,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:41:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:41:58,416 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:58,417 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:41:58,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:41:58,417 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,059 - lava_callback - INFO - Received callback for node 698295c93507dcddca58e1de 2026-02-06 00:41:59,064 - lava_callback - INFO - Starting async job submit for node 698295c93507dcddca58e1de INFO: 172.19.0.1:49548 - "POST /node/698295c93507dcddca58e1de HTTP/1.0" 202 Accepted 2026-02-06 00:41:59,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:41:59,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:41:59,080 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,080 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:41:59,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:41:59,081 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,727 - lava_callback - INFO - Received callback for node 698295c93507dcddca58e1e0 2026-02-06 00:41:59,732 - lava_callback - INFO - Starting async job submit for node 698295c93507dcddca58e1e0 INFO: 172.19.0.1:56020 - "POST /node/698295c93507dcddca58e1e0 HTTP/1.0" 202 Accepted 2026-02-06 00:41:59,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:41:59,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:41:59,747 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:41:59,747 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:41:59,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:41:59,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:00,418 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:00,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:00,419 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:01,081 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:01,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:01,082 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:01,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:01,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:01,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:04,419 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:04,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:04,420 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:05,082 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:05,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:05,084 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:05,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:05,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:05,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:56034 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:12,421 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:12,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:12,422 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1dc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:13,084 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:13,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:13,085 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1de'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:13,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:13,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:13,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295c93507dcddca58e1e0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:48380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59752 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:28,423 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1dc 2026-02-06 00:42:28,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1dc HTTP/1.1" 502 150 2026-02-06 00:42:28,425 - lava_callback - ERROR - Error processing callback for node 698295c93507dcddca58e1dc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1dc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1dc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1dc (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:42:29,085 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1de 2026-02-06 00:42:29,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1de HTTP/1.1" 502 150 2026-02-06 00:42:29,087 - lava_callback - ERROR - Error processing callback for node 698295c93507dcddca58e1de: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1de (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1de (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1de (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:42:29,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295c93507dcddca58e1e0 2026-02-06 00:42:29,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295c93507dcddca58e1e0 HTTP/1.1" 502 150 2026-02-06 00:42:29,754 - lava_callback - ERROR - Error processing callback for node 698295c93507dcddca58e1e0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1e0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1e0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295c93507dcddca58e1e0 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:38360 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:38,300 - lava_callback - INFO - Received callback for node 698295f93507dcddca58e1eb INFO: 172.19.0.1:38374 - "POST /node/698295f93507dcddca58e1eb HTTP/1.0" 202 Accepted 2026-02-06 00:42:38,306 - lava_callback - INFO - Starting async job submit for node 698295f93507dcddca58e1eb 2026-02-06 00:42:38,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:42:38,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:38,328 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:38,328 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:38,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:38,329 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:38,986 - lava_callback - INFO - Received callback for node 698295fa3507dcddca58e1ed INFO: 172.19.0.1:38386 - "POST /node/698295fa3507dcddca58e1ed HTTP/1.0" 202 Accepted 2026-02-06 00:42:38,992 - lava_callback - INFO - Starting async job submit for node 698295fa3507dcddca58e1ed 2026-02-06 00:42:38,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:42:39,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:39,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:39,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:39,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:39,047 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:39,631 - lava_callback - INFO - Received callback for node 698295fa3507dcddca58e1ef INFO: 172.19.0.1:38400 - "POST /node/698295fa3507dcddca58e1ef HTTP/1.0" 202 Accepted 2026-02-06 00:42:39,637 - lava_callback - INFO - Starting async job submit for node 698295fa3507dcddca58e1ef 2026-02-06 00:42:39,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:42:39,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:39,650 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:39,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:39,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:39,651 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:40,329 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:40,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:40,331 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:41,047 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:41,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:41,049 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:41,651 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:41,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:41,653 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:44,331 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:44,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:44,334 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:45,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:45,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:45,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:45,653 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:45,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:45,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:60816 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:42:52,335 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:42:52,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:42:52,337 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295f93507dcddca58e1eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:53,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:42:53,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:42:53,052 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:42:53,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:42:53,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:42:53,656 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698295fa3507dcddca58e1ef'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42864 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:08,337 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295f93507dcddca58e1eb 2026-02-06 00:43:08,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295f93507dcddca58e1eb HTTP/1.1" 502 150 2026-02-06 00:43:08,339 - lava_callback - ERROR - Error processing callback for node 698295f93507dcddca58e1eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295f93507dcddca58e1eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295f93507dcddca58e1eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295f93507dcddca58e1eb (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:43:09,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ed 2026-02-06 00:43:09,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ed HTTP/1.1" 502 150 2026-02-06 00:43:09,054 - lava_callback - ERROR - Error processing callback for node 698295fa3507dcddca58e1ed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ed (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:43:09,656 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698295fa3507dcddca58e1ef 2026-02-06 00:43:09,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698295fa3507dcddca58e1ef HTTP/1.1" 502 150 2026-02-06 00:43:09,658 - lava_callback - ERROR - Error processing callback for node 698295fa3507dcddca58e1ef: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ef (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ef (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698295fa3507dcddca58e1ef (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:42514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42518 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:18,351 - lava_callback - INFO - Received callback for node 698296243507dcddca58e1f6 INFO: 172.19.0.1:42530 - "POST /node/698296243507dcddca58e1f6 HTTP/1.0" 202 Accepted 2026-02-06 00:43:18,357 - lava_callback - INFO - Starting async job submit for node 698296243507dcddca58e1f6 2026-02-06 00:43:18,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:43:18,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:18,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:18,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:18,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:18,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:20,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:20,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:20,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:24,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:24,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:24,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:37570 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:32,374 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:32,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:32,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296243507dcddca58e1f6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46462 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:38,326 - lava_callback - INFO - Received callback for node 698296253507dcddca58e1f8 INFO: 172.19.0.1:46472 - "POST /node/698296253507dcddca58e1f8 HTTP/1.0" 202 Accepted 2026-02-06 00:43:38,332 - lava_callback - INFO - Starting async job submit for node 698296253507dcddca58e1f8 2026-02-06 00:43:38,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:43:38,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:38,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:38,345 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:38,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:38,345 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:39,012 - lava_callback - INFO - Received callback for node 698296253507dcddca58e1fa INFO: 172.19.0.1:46480 - "POST /node/698296253507dcddca58e1fa HTTP/1.0" 202 Accepted 2026-02-06 00:43:39,018 - lava_callback - INFO - Starting async job submit for node 698296253507dcddca58e1fa 2026-02-06 00:43:39,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:43:39,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:39,030 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:39,031 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:39,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:39,031 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:40,346 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:40,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:40,347 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:41,032 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:41,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:41,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:44,347 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:44,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:44,349 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:45,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:45,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:45,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:49112 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:43:48,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296243507dcddca58e1f6 2026-02-06 00:43:48,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296243507dcddca58e1f6 HTTP/1.1" 502 150 2026-02-06 00:43:48,378 - lava_callback - ERROR - Error processing callback for node 698296243507dcddca58e1f6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296243507dcddca58e1f6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296243507dcddca58e1f6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296243507dcddca58e1f6 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:43:52,349 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:43:52,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:43:52,350 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:43:53,035 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:43:53,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:43:53,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296253507dcddca58e1fa'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51244 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:08,350 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1f8 2026-02-06 00:44:08,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1f8 HTTP/1.1" 502 150 2026-02-06 00:44:08,352 - lava_callback - ERROR - Error processing callback for node 698296253507dcddca58e1f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1f8 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:44:09,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296253507dcddca58e1fa 2026-02-06 00:44:09,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296253507dcddca58e1fa HTTP/1.1" 502 150 2026-02-06 00:44:09,037 - lava_callback - ERROR - Error processing callback for node 698296253507dcddca58e1fa: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1fa (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1fa (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296253507dcddca58e1fa (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:43800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43832 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:18,372 - lava_callback - INFO - Received callback for node 698296503507dcddca58e202 INFO: 172.19.0.1:43836 - "POST /node/698296503507dcddca58e202 HTTP/1.0" 202 Accepted 2026-02-06 00:44:18,378 - lava_callback - INFO - Starting async job submit for node 698296503507dcddca58e202 2026-02-06 00:44:18,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:18,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:18,392 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:18,392 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:18,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:18,393 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,272 - lava_callback - INFO - Received callback for node 698296503507dcddca58e204 INFO: 172.19.0.1:43850 - "POST /node/698296503507dcddca58e204 HTTP/1.0" 202 Accepted 2026-02-06 00:44:19,278 - lava_callback - INFO - Starting async job submit for node 698296503507dcddca58e204 2026-02-06 00:44:19,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:19,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:19,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,319 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:19,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:19,319 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,917 - lava_callback - INFO - Received callback for node 698296503507dcddca58e206 INFO: 172.19.0.1:55108 - "POST /node/698296503507dcddca58e206 HTTP/1.0" 202 Accepted 2026-02-06 00:44:19,923 - lava_callback - INFO - Starting async job submit for node 698296503507dcddca58e206 2026-02-06 00:44:19,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:19,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:19,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:19,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:19,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:19,944 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:20,393 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:20,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:20,394 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:21,320 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:21,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:21,321 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:21,944 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:21,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:21,946 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:24,395 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:24,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:24,396 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:25,321 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:25,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:25,322 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:25,946 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:25,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:25,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:55112 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:32,396 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:32,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:32,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e202'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:33,323 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:33,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:33,324 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e204'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:33,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:33,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296503507dcddca58e206'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40002 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:48,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e202 2026-02-06 00:44:48,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e202 HTTP/1.1" 502 150 2026-02-06 00:44:48,399 - lava_callback - ERROR - Error processing callback for node 698296503507dcddca58e202: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e202 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e202 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e202 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:44:49,324 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e204 2026-02-06 00:44:49,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e204 HTTP/1.1" 502 150 2026-02-06 00:44:49,326 - lava_callback - ERROR - Error processing callback for node 698296503507dcddca58e204: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e204 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e204 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e204 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:44:49,949 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296503507dcddca58e206 2026-02-06 00:44:49,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296503507dcddca58e206 HTTP/1.1" 502 150 2026-02-06 00:44:49,950 - lava_callback - ERROR - Error processing callback for node 698296503507dcddca58e206: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e206 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e206 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296503507dcddca58e206 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:59658 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:44:58,411 - lava_callback - INFO - Received callback for node 6982967f3507dcddca58e20f INFO: 172.19.0.1:59666 - "POST /node/6982967f3507dcddca58e20f HTTP/1.0" 202 Accepted 2026-02-06 00:44:58,416 - lava_callback - INFO - Starting async job submit for node 6982967f3507dcddca58e20f 2026-02-06 00:44:58,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:58,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:44:58,452 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:58,452 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:44:58,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:44:58,453 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,055 - lava_callback - INFO - Received callback for node 698296803507dcddca58e211 INFO: 172.19.0.1:59670 - "POST /node/698296803507dcddca58e211 HTTP/1.0" 202 Accepted 2026-02-06 00:44:59,060 - lava_callback - INFO - Starting async job submit for node 698296803507dcddca58e211 2026-02-06 00:44:59,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:59,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:44:59,095 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,095 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:44:59,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:44:59,096 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,698 - lava_callback - INFO - Received callback for node 698296803507dcddca58e213 INFO: 172.19.0.1:45802 - "POST /node/698296803507dcddca58e213 HTTP/1.0" 202 Accepted 2026-02-06 00:44:59,704 - lava_callback - INFO - Starting async job submit for node 698296803507dcddca58e213 2026-02-06 00:44:59,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:44:59,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:44:59,716 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:44:59,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:44:59,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:44:59,717 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:00,453 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:00,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:00,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:01,096 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:01,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:01,097 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:01,717 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:01,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:01,719 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:04,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:04,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:04,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:05,098 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:05,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:05,099 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:05,719 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:05,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:05,720 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:45818 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:45:12,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:12,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:12,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6982967f3507dcddca58e20f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:13,099 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:13,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:13,101 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e211'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:13,721 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:13,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:13,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296803507dcddca58e213'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:48642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36126 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:45:28,458 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6982967f3507dcddca58e20f 2026-02-06 00:45:28,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6982967f3507dcddca58e20f HTTP/1.1" 502 150 2026-02-06 00:45:28,459 - lava_callback - ERROR - Error processing callback for node 6982967f3507dcddca58e20f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6982967f3507dcddca58e20f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6982967f3507dcddca58e20f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6982967f3507dcddca58e20f (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:45:29,101 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e211 2026-02-06 00:45:29,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e211 HTTP/1.1" 502 150 2026-02-06 00:45:29,103 - lava_callback - ERROR - Error processing callback for node 698296803507dcddca58e211: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e211 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e211 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e211 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:45:29,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296803507dcddca58e213 2026-02-06 00:45:29,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296803507dcddca58e213 HTTP/1.1" 502 150 2026-02-06 00:45:29,723 - lava_callback - ERROR - Error processing callback for node 698296803507dcddca58e213: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e213 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e213 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296803507dcddca58e213 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48686 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:45:58,693 - lava_callback - INFO - Received callback for node 698296b13507dcddca58e27e INFO: 172.19.0.1:48696 - "POST /node/698296b13507dcddca58e27e HTTP/1.0" 202 Accepted 2026-02-06 00:45:58,698 - lava_callback - INFO - Starting async job submit for node 698296b13507dcddca58e27e 2026-02-06 00:45:58,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:45:58,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:45:58,722 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:58,722 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:45:58,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:45:58,723 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:59,352 - lava_callback - INFO - Received callback for node 698296b23507dcddca58e280 INFO: 172.19.0.1:48708 - "POST /node/698296b23507dcddca58e280 HTTP/1.0" 202 Accepted 2026-02-06 00:45:59,359 - lava_callback - INFO - Starting async job submit for node 698296b23507dcddca58e280 2026-02-06 00:45:59,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:45:59,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:45:59,397 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:59,397 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:45:59,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:45:59,398 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:45:59,996 - lava_callback - INFO - Received callback for node 698296b23507dcddca58e282 INFO: 172.19.0.1:48624 - "POST /node/698296b23507dcddca58e282 HTTP/1.0" 202 Accepted 2026-02-06 00:46:00,006 - lava_callback - INFO - Starting async job submit for node 698296b23507dcddca58e282 2026-02-06 00:46:00,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:46:00,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:00,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:00,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:00,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:00,020 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:00,723 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:00,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:00,725 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:01,398 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:01,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:01,400 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:02,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:02,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:02,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:04,725 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:04,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:04,733 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:05,401 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:05,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:05,404 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:06,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:06,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:06,025 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:48628 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:46:12,733 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:12,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:12,735 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b13507dcddca58e27e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:13,405 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:13,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:13,406 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e280'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:46:14,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:14,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:14,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698296b23507dcddca58e282'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50850 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:46:28,735 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b13507dcddca58e27e 2026-02-06 00:46:28,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b13507dcddca58e27e HTTP/1.1" 502 150 2026-02-06 00:46:28,737 - lava_callback - ERROR - Error processing callback for node 698296b13507dcddca58e27e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b13507dcddca58e27e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b13507dcddca58e27e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b13507dcddca58e27e (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:46:29,406 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e280 2026-02-06 00:46:29,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e280 HTTP/1.1" 502 150 2026-02-06 00:46:29,408 - lava_callback - ERROR - Error processing callback for node 698296b23507dcddca58e280: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e280 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e280 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e280 (Caused by ResponseError('too many 502 error responses')) 2026-02-06 00:46:30,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698296b23507dcddca58e282 2026-02-06 00:46:30,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698296b23507dcddca58e282 HTTP/1.1" 502 150 2026-02-06 00:46:30,029 - lava_callback - ERROR - Error processing callback for node 698296b23507dcddca58e282: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e282 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e282 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698296b23507dcddca58e282 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49106 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:52:11,174 - lava_callback - INFO - Received callback for node 698455b73a8748f28208488e INFO: 172.19.0.1:54282 - "POST /node/698455b73a8748f28208488e HTTP/1.0" 202 Accepted 2026-02-06 00:52:11,182 - lava_callback - INFO - Starting async job submit for node 698455b73a8748f28208488e 2026-02-06 00:52:11,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 00:52:11,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:11,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:52:11,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:11,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:11,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:52:13,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:13,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:13,240 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:54284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54314 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:52:17,240 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:17,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:17,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 00:52:25,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:25,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:25,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b73a8748f28208488e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50612 - "GET / HTTP/1.0" 200 OK 2026-02-06 00:52:41,243 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b73a8748f28208488e 2026-02-06 00:52:41,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b73a8748f28208488e HTTP/1.1" 502 150 2026-02-06 00:52:41,245 - lava_callback - ERROR - Error processing callback for node 698455b73a8748f28208488e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b73a8748f28208488e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b73a8748f28208488e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b73a8748f28208488e (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43412 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:12:55,866 - lava_callback - INFO - Received callback for node 698455b83a8748f282084891 INFO: 172.19.0.1:47382 - "POST /node/698455b83a8748f282084891 HTTP/1.0" 202 Accepted 2026-02-06 01:12:55,875 - lava_callback - INFO - Starting async job submit for node 698455b83a8748f282084891 2026-02-06 01:12:55,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 01:12:55,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:12:55,937 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:12:55,937 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:12:55,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:12:55,938 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:47390 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:12:57,938 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:12:57,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:12:57,939 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:13:01,939 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:13:01,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:13:01,941 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:47514 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:13:09,941 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:13:09,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:13:09,943 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b83a8748f282084891'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53244 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:13:25,943 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b83a8748f282084891 2026-02-06 01:13:25,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b83a8748f282084891 HTTP/1.1" 502 150 2026-02-06 01:13:25,951 - lava_callback - ERROR - Error processing callback for node 698455b83a8748f282084891: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b83a8748f282084891 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b83a8748f282084891 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b83a8748f282084891 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60128 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:54:19,800 - lava_callback - INFO - Received callback for node 698455b93a8748f282084894 INFO: 172.19.0.1:34902 - "POST /node/698455b93a8748f282084894 HTTP/1.0" 202 Accepted 2026-02-06 01:54:19,808 - lava_callback - INFO - Starting async job submit for node 698455b93a8748f282084894 2026-02-06 01:54:19,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 01:54:19,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:19,891 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:54:19,891 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:19,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:19,892 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:54:21,892 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:21,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:21,893 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 01:54:25,893 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:25,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:25,895 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:34908 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:54:33,895 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:33,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:33,897 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698455b93a8748f282084894'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:32856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59290 - "GET / HTTP/1.0" 200 OK 2026-02-06 01:54:49,897 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698455b93a8748f282084894 2026-02-06 01:54:49,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698455b93a8748f282084894 HTTP/1.1" 502 150 2026-02-06 01:54:49,900 - lava_callback - ERROR - Error processing callback for node 698455b93a8748f282084894: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b93a8748f282084894 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b93a8748f282084894 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698455b93a8748f282084894 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:34080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59026 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:14:49,160 - lava_callback - INFO - Received callback for node 698456903a8748f2820848f5 INFO: 172.19.0.1:59038 - "POST /node/698456903a8748f2820848f5 HTTP/1.0" 202 Accepted 2026-02-06 02:14:49,168 - lava_callback - INFO - Starting async job submit for node 698456903a8748f2820848f5 2026-02-06 02:14:49,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 02:14:49,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:49,224 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:14:49,224 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:14:49,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:49,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:14:51,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:14:51,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:51,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:14:55,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:14:55,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:14:55,235 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:42582 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:15:03,235 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:15:03,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:15:03,237 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456903a8748f2820848f5'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38970 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:15:19,237 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456903a8748f2820848f5 2026-02-06 02:15:19,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456903a8748f2820848f5 HTTP/1.1" 502 150 2026-02-06 02:15:19,238 - lava_callback - ERROR - Error processing callback for node 698456903a8748f2820848f5: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456903a8748f2820848f5 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456903a8748f2820848f5 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456903a8748f2820848f5 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37406 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:35:42,066 - lava_callback - INFO - Received callback for node 698456913a8748f2820848f8 INFO: 172.19.0.1:46714 - "POST /node/698456913a8748f2820848f8 HTTP/1.0" 202 Accepted 2026-02-06 02:35:42,079 - lava_callback - INFO - Starting async job submit for node 698456913a8748f2820848f8 2026-02-06 02:35:42,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 02:35:42,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:42,212 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:35:42,212 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:42,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:42,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:35:44,214 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:44,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:44,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46726 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:35:48,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:48,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:48,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:35:56,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:35:56,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:35:56,219 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456913a8748f2820848f8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52084 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:36:12,219 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456913a8748f2820848f8 2026-02-06 02:36:12,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456913a8748f2820848f8 HTTP/1.1" 502 150 2026-02-06 02:36:12,221 - lava_callback - ERROR - Error processing callback for node 698456913a8748f2820848f8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456913a8748f2820848f8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456913a8748f2820848f8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456913a8748f2820848f8 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:53214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37020 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:22,931 - lava_callback - INFO - Received callback for node 698456953a8748f2820848fb INFO: 172.19.0.1:36956 - "POST /node/698456953a8748f2820848fb HTTP/1.0" 202 Accepted 2026-02-06 02:56:22,938 - lava_callback - INFO - Starting async job submit for node 698456953a8748f2820848fb 2026-02-06 02:56:22,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 02:56:22,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:22,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:56:22,994 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:22,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:22,994 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 02:56:24,995 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:24,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:24,996 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:36964 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:28,996 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:28,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:28,998 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:33416 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:36,998 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:36,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:36,999 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456953a8748f2820848fb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:51256 - "GET / HTTP/1.0" 200 OK 2026-02-06 02:56:53,000 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456953a8748f2820848fb 2026-02-06 02:56:53,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456953a8748f2820848fb HTTP/1.1" 502 150 2026-02-06 02:56:53,001 - lava_callback - ERROR - Error processing callback for node 698456953a8748f2820848fb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456953a8748f2820848fb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456953a8748f2820848fb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456953a8748f2820848fb (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:41052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48434 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:02,574 - lava_callback - INFO - Received callback for node 698456fc3a8748f2820849a6 INFO: 172.19.0.1:41200 - "POST /node/698456fc3a8748f2820849a6 HTTP/1.0" 202 Accepted 2026-02-06 03:17:02,582 - lava_callback - INFO - Starting async job submit for node 698456fc3a8748f2820849a6 2026-02-06 03:17:02,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 03:17:02,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:02,638 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:17:02,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:02,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:02,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:17:04,639 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:04,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:04,640 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:41212 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:08,641 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:08,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:08,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53450 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:16,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:16,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:16,644 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456fc3a8748f2820849a6'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54324 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:17:32,644 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456fc3a8748f2820849a6 2026-02-06 03:17:32,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456fc3a8748f2820849a6 HTTP/1.1" 502 150 2026-02-06 03:17:32,649 - lava_callback - ERROR - Error processing callback for node 698456fc3a8748f2820849a6: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456fc3a8748f2820849a6 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456fc3a8748f2820849a6 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456fc3a8748f2820849a6 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33080 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:37:45,174 - lava_callback - INFO - Received callback for node 698456ff3a8748f2820849c7 INFO: 172.19.0.1:57786 - "POST /node/698456ff3a8748f2820849c7 HTTP/1.0" 202 Accepted 2026-02-06 03:37:45,182 - lava_callback - INFO - Starting async job submit for node 698456ff3a8748f2820849c7 2026-02-06 03:37:45,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 03:37:45,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:45,249 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:37:45,249 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:45,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:45,250 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:57800 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:37:47,250 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:47,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:47,251 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:37:51,252 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:51,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:51,253 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:41428 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:37:59,253 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:37:59,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:37:59,255 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698456ff3a8748f2820849c7'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57156 - "GET /api/metrics HTTP/1.0" 200 OK 2026-02-06 03:38:15,255 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698456ff3a8748f2820849c7 2026-02-06 03:38:15,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698456ff3a8748f2820849c7 HTTP/1.1" 502 150 2026-02-06 03:38:15,256 - lava_callback - ERROR - Error processing callback for node 698456ff3a8748f2820849c7: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456ff3a8748f2820849c7 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456ff3a8748f2820849c7 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698456ff3a8748f2820849c7 (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38064 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:16,830 - lava_callback - INFO - Received callback for node 698457033a8748f2820849fd INFO: 172.19.0.1:38076 - "POST /node/698457033a8748f2820849fd HTTP/1.0" 202 Accepted 2026-02-06 03:58:16,838 - lava_callback - INFO - Starting async job submit for node 698457033a8748f2820849fd 2026-02-06 03:58:16,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 03:58:16,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:16,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:58:16,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:16,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:16,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 03:58:18,900 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:18,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:18,901 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:38082 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:22,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:22,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:22,903 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:36396 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:30,904 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:30,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:30,905 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698457033a8748f2820849fd'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43496 - "GET / HTTP/1.0" 200 OK 2026-02-06 03:58:46,905 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698457033a8748f2820849fd 2026-02-06 03:58:46,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698457033a8748f2820849fd HTTP/1.1" 502 150 2026-02-06 03:58:46,907 - lava_callback - ERROR - Error processing callback for node 698457033a8748f2820849fd: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698457033a8748f2820849fd (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698457033a8748f2820849fd (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698457033a8748f2820849fd (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:43376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44116 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:10,546 - lava_callback - INFO - Received callback for node 698458523a8748f282084e0e INFO: 172.19.0.1:34724 - "POST /node/698458523a8748f282084e0e HTTP/1.0" 202 Accepted 2026-02-06 04:19:10,553 - lava_callback - INFO - Starting async job submit for node 698458523a8748f282084e0e 2026-02-06 04:19:10,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 04:19:10,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:10,631 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:19:10,631 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:10,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:10,632 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:19:12,632 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:12,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:12,637 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:34726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34742 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:16,638 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:16,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:16,639 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:39694 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:24,640 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:24,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:24,642 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458523a8748f282084e0e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35864 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:19:40,642 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458523a8748f282084e0e 2026-02-06 04:19:40,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458523a8748f282084e0e HTTP/1.1" 502 150 2026-02-06 04:19:40,644 - lava_callback - ERROR - Error processing callback for node 698458523a8748f282084e0e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458523a8748f282084e0e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458523a8748f282084e0e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458523a8748f282084e0e (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35892 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:03,055 - lava_callback - INFO - Received callback for node 698458603a8748f282084e4a INFO: 172.19.0.1:46770 - "POST /node/698458603a8748f282084e4a HTTP/1.0" 202 Accepted 2026-02-06 04:40:03,063 - lava_callback - INFO - Starting async job submit for node 698458603a8748f282084e4a 2026-02-06 04:40:03,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-06 04:40:03,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:03,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:40:03,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:03,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:03,124 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-02-06 04:40:05,124 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:05,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:05,125 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:46786 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:09,126 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:09,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:09,127 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:53074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53090 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:17,127 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:17,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:17,133 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/698458603a8748f282084e4a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.19.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49334 - "GET / HTTP/1.0" 200 OK 2026-02-06 04:40:33,134 - urllib3.connectionpool - DEBUG - Retry: /latest/node/698458603a8748f282084e4a 2026-02-06 04:40:33,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/698458603a8748f282084e4a HTTP/1.1" 502 150 2026-02-06 04:40:33,135 - lava_callback - ERROR - Error processing callback for node 698458603a8748f282084e4a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458603a8748f282084e4a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458603a8748f282084e4a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 286, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/698458603a8748f282084e4a (Caused by ResponseError('too many 502 error responses')) INFO: 172.19.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]