INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59398 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:39:59,849 - lava_callback - INFO - Received callback for node 69620249707d9db6aa58cfde 2026-01-12 07:39:59,855 - lava_callback - INFO - Starting async job submit for node 69620249707d9db6aa58cfde INFO: 172.20.0.1:42478 - "POST /node/69620249707d9db6aa58cfde HTTP/1.0" 202 Accepted 2026-01-12 07:39:59,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 07:39:59,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620249707d9db6aa58cfde HTTP/1.1" 502 150 2026-01-12 07:39:59,872 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620249707d9db6aa58cfde'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:39:59,872 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620249707d9db6aa58cfde 2026-01-12 07:39:59,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620249707d9db6aa58cfde HTTP/1.1" 502 150 2026-01-12 07:39:59,873 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620249707d9db6aa58cfde'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:40:01,873 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620249707d9db6aa58cfde 2026-01-12 07:40:01,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620249707d9db6aa58cfde HTTP/1.1" 502 150 2026-01-12 07:40:01,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620249707d9db6aa58cfde'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:40:05,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620249707d9db6aa58cfde 2026-01-12 07:40:05,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620249707d9db6aa58cfde HTTP/1.1" 502 150 2026-01-12 07:40:05,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620249707d9db6aa58cfde'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:42482 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:40:13,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620249707d9db6aa58cfde 2026-01-12 07:40:13,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620249707d9db6aa58cfde HTTP/1.1" 502 150 2026-01-12 07:40:13,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620249707d9db6aa58cfde'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:49862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52534 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:40:29,878 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620249707d9db6aa58cfde 2026-01-12 07:40:29,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620249707d9db6aa58cfde HTTP/1.1" 502 150 2026-01-12 07:40:29,880 - lava_callback - ERROR - Error processing callback for node 69620249707d9db6aa58cfde: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620249707d9db6aa58cfde (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620249707d9db6aa58cfde (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620249707d9db6aa58cfde (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41634 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:41:39,930 - lava_callback - INFO - Received callback for node 6962029f707d9db6aa58cfe3 INFO: 172.20.0.1:32902 - "POST /node/6962029f707d9db6aa58cfe3 HTTP/1.0" 202 Accepted 2026-01-12 07:41:39,936 - lava_callback - INFO - Starting async job submit for node 6962029f707d9db6aa58cfe3 2026-01-12 07:41:39,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 07:41:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962029f707d9db6aa58cfe3 HTTP/1.1" 502 150 2026-01-12 07:41:39,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962029f707d9db6aa58cfe3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:41:39,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962029f707d9db6aa58cfe3 2026-01-12 07:41:39,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962029f707d9db6aa58cfe3 HTTP/1.1" 502 150 2026-01-12 07:41:39,950 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962029f707d9db6aa58cfe3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:41:41,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962029f707d9db6aa58cfe3 2026-01-12 07:41:41,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962029f707d9db6aa58cfe3 HTTP/1.1" 502 150 2026-01-12 07:41:41,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962029f707d9db6aa58cfe3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:41:45,952 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962029f707d9db6aa58cfe3 2026-01-12 07:41:45,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962029f707d9db6aa58cfe3 HTTP/1.1" 502 150 2026-01-12 07:41:45,954 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962029f707d9db6aa58cfe3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:32918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41512 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:41:53,955 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962029f707d9db6aa58cfe3 2026-01-12 07:41:53,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962029f707d9db6aa58cfe3 HTTP/1.1" 502 150 2026-01-12 07:41:53,956 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962029f707d9db6aa58cfe3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46380 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:42:09,956 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962029f707d9db6aa58cfe3 2026-01-12 07:42:09,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962029f707d9db6aa58cfe3 HTTP/1.1" 502 150 2026-01-12 07:42:09,958 - lava_callback - ERROR - Error processing callback for node 6962029f707d9db6aa58cfe3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962029f707d9db6aa58cfe3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962029f707d9db6aa58cfe3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962029f707d9db6aa58cfe3 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:51790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54728 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:42:59,957 - lava_callback - INFO - Received callback for node 69620300707d9db6aa58cfe8 INFO: 172.20.0.1:46238 - "POST /node/69620300707d9db6aa58cfe8 HTTP/1.0" 202 Accepted 2026-01-12 07:42:59,962 - lava_callback - INFO - Starting async job submit for node 69620300707d9db6aa58cfe8 2026-01-12 07:42:59,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 07:42:59,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620300707d9db6aa58cfe8 HTTP/1.1" 502 150 2026-01-12 07:42:59,975 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620300707d9db6aa58cfe8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:42:59,975 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620300707d9db6aa58cfe8 2026-01-12 07:42:59,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620300707d9db6aa58cfe8 HTTP/1.1" 502 150 2026-01-12 07:42:59,976 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620300707d9db6aa58cfe8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:43:01,976 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620300707d9db6aa58cfe8 2026-01-12 07:43:01,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620300707d9db6aa58cfe8 HTTP/1.1" 502 150 2026-01-12 07:43:01,978 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620300707d9db6aa58cfe8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:43:05,978 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620300707d9db6aa58cfe8 2026-01-12 07:43:05,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620300707d9db6aa58cfe8 HTTP/1.1" 502 150 2026-01-12 07:43:05,979 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620300707d9db6aa58cfe8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:46250 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:43:13,980 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620300707d9db6aa58cfe8 2026-01-12 07:43:13,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620300707d9db6aa58cfe8 HTTP/1.1" 502 150 2026-01-12 07:43:13,981 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620300707d9db6aa58cfe8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:57644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40156 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:43:29,981 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620300707d9db6aa58cfe8 2026-01-12 07:43:29,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620300707d9db6aa58cfe8 HTTP/1.1" 502 150 2026-01-12 07:43:29,983 - lava_callback - ERROR - Error processing callback for node 69620300707d9db6aa58cfe8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620300707d9db6aa58cfe8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620300707d9db6aa58cfe8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620300707d9db6aa58cfe8 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:44:40,091 - lava_callback - INFO - Received callback for node 69620365707d9db6aa58cfed INFO: 172.20.0.1:50392 - "POST /node/69620365707d9db6aa58cfed HTTP/1.0" 202 Accepted 2026-01-12 07:44:40,097 - lava_callback - INFO - Starting async job submit for node 69620365707d9db6aa58cfed 2026-01-12 07:44:40,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 07:44:40,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620365707d9db6aa58cfed HTTP/1.1" 502 150 2026-01-12 07:44:40,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620365707d9db6aa58cfed'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:44:40,117 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620365707d9db6aa58cfed 2026-01-12 07:44:40,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620365707d9db6aa58cfed HTTP/1.1" 502 150 2026-01-12 07:44:40,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620365707d9db6aa58cfed'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:44:42,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620365707d9db6aa58cfed 2026-01-12 07:44:42,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620365707d9db6aa58cfed HTTP/1.1" 502 150 2026-01-12 07:44:42,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620365707d9db6aa58cfed'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:44:46,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620365707d9db6aa58cfed 2026-01-12 07:44:46,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620365707d9db6aa58cfed HTTP/1.1" 502 150 2026-01-12 07:44:46,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620365707d9db6aa58cfed'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49256 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:44:54,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620365707d9db6aa58cfed 2026-01-12 07:44:54,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620365707d9db6aa58cfed HTTP/1.1" 502 150 2026-01-12 07:44:54,122 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620365707d9db6aa58cfed'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40682 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:45:10,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620365707d9db6aa58cfed 2026-01-12 07:45:10,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620365707d9db6aa58cfed HTTP/1.1" 502 150 2026-01-12 07:45:10,124 - lava_callback - ERROR - Error processing callback for node 69620365707d9db6aa58cfed: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620365707d9db6aa58cfed (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620365707d9db6aa58cfed (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620365707d9db6aa58cfed (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:40790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45892 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:48:00,265 - lava_callback - INFO - Received callback for node 6962042a707d9db6aa58d004 INFO: 172.20.0.1:50596 - "POST /node/6962042a707d9db6aa58d004 HTTP/1.0" 202 Accepted 2026-01-12 07:48:00,275 - lava_callback - INFO - Starting async job submit for node 6962042a707d9db6aa58d004 2026-01-12 07:48:00,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 07:48:00,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962042a707d9db6aa58d004 HTTP/1.1" 502 150 2026-01-12 07:48:00,482 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962042a707d9db6aa58d004'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:48:00,482 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962042a707d9db6aa58d004 2026-01-12 07:48:00,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962042a707d9db6aa58d004 HTTP/1.1" 502 150 2026-01-12 07:48:00,483 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962042a707d9db6aa58d004'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:48:02,483 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962042a707d9db6aa58d004 2026-01-12 07:48:02,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962042a707d9db6aa58d004 HTTP/1.1" 502 150 2026-01-12 07:48:02,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962042a707d9db6aa58d004'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:50612 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:48:06,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962042a707d9db6aa58d004 2026-01-12 07:48:06,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962042a707d9db6aa58d004 HTTP/1.1" 502 150 2026-01-12 07:48:06,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962042a707d9db6aa58d004'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:48:14,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962042a707d9db6aa58d004 2026-01-12 07:48:14,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962042a707d9db6aa58d004 HTTP/1.1" 502 150 2026-01-12 07:48:14,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962042a707d9db6aa58d004'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:49882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46580 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:48:30,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962042a707d9db6aa58d004 2026-01-12 07:48:30,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962042a707d9db6aa58d004 HTTP/1.1" 502 150 2026-01-12 07:48:30,493 - lava_callback - ERROR - Error processing callback for node 6962042a707d9db6aa58d004: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962042a707d9db6aa58d004 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962042a707d9db6aa58d004 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962042a707d9db6aa58d004 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43646 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:49:20,350 - lava_callback - INFO - Received callback for node 6962047d707d9db6aa58d00c INFO: 172.20.0.1:52322 - "POST /node/6962047d707d9db6aa58d00c HTTP/1.0" 202 Accepted 2026-01-12 07:49:20,355 - lava_callback - INFO - Starting async job submit for node 6962047d707d9db6aa58d00c 2026-01-12 07:49:20,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 07:49:20,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962047d707d9db6aa58d00c HTTP/1.1" 502 150 2026-01-12 07:49:20,369 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962047d707d9db6aa58d00c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:49:20,369 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962047d707d9db6aa58d00c 2026-01-12 07:49:20,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962047d707d9db6aa58d00c HTTP/1.1" 502 150 2026-01-12 07:49:20,370 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962047d707d9db6aa58d00c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:49:22,370 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962047d707d9db6aa58d00c 2026-01-12 07:49:22,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962047d707d9db6aa58d00c HTTP/1.1" 502 150 2026-01-12 07:49:22,372 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962047d707d9db6aa58d00c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:49:26,372 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962047d707d9db6aa58d00c 2026-01-12 07:49:26,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962047d707d9db6aa58d00c HTTP/1.1" 502 150 2026-01-12 07:49:26,373 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962047d707d9db6aa58d00c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:52328 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:49:34,373 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962047d707d9db6aa58d00c 2026-01-12 07:49:34,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962047d707d9db6aa58d00c HTTP/1.1" 502 150 2026-01-12 07:49:34,375 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962047d707d9db6aa58d00c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:49:50,375 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962047d707d9db6aa58d00c 2026-01-12 07:49:50,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962047d707d9db6aa58d00c HTTP/1.1" 502 150 2026-01-12 07:49:50,376 - lava_callback - ERROR - Error processing callback for node 6962047d707d9db6aa58d00c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962047d707d9db6aa58d00c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962047d707d9db6aa58d00c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962047d707d9db6aa58d00c (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:54326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51298 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:53:49,062 - lava_callback - INFO - Received callback for node 696431a81d3b6d6d76a05f3e INFO: 172.20.0.1:51300 - "POST /node/696431a81d3b6d6d76a05f3e HTTP/1.0" 202 Accepted 2026-01-12 07:53:49,073 - lava_callback - INFO - Starting async job submit for node 696431a81d3b6d6d76a05f3e 2026-01-12 07:53:49,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 07:53:49,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696431a81d3b6d6d76a05f3e HTTP/1.1" 502 150 2026-01-12 07:53:49,141 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696431a81d3b6d6d76a05f3e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:53:49,141 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696431a81d3b6d6d76a05f3e 2026-01-12 07:53:49,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696431a81d3b6d6d76a05f3e HTTP/1.1" 502 150 2026-01-12 07:53:49,142 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696431a81d3b6d6d76a05f3e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 07:53:51,142 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696431a81d3b6d6d76a05f3e 2026-01-12 07:53:51,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696431a81d3b6d6d76a05f3e HTTP/1.1" 502 150 2026-01-12 07:53:51,143 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696431a81d3b6d6d76a05f3e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:57242 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:53:55,144 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696431a81d3b6d6d76a05f3e 2026-01-12 07:53:55,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696431a81d3b6d6d76a05f3e HTTP/1.1" 502 150 2026-01-12 07:53:55,145 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696431a81d3b6d6d76a05f3e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:57258 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:54:03,145 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696431a81d3b6d6d76a05f3e 2026-01-12 07:54:03,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696431a81d3b6d6d76a05f3e HTTP/1.1" 502 150 2026-01-12 07:54:03,146 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696431a81d3b6d6d76a05f3e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46576 - "GET / HTTP/1.0" 200 OK 2026-01-12 07:54:19,146 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696431a81d3b6d6d76a05f3e 2026-01-12 07:54:19,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696431a81d3b6d6d76a05f3e HTTP/1.1" 502 150 2026-01-12 07:54:19,148 - lava_callback - ERROR - Error processing callback for node 696431a81d3b6d6d76a05f3e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696431a81d3b6d6d76a05f3e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696431a81d3b6d6d76a05f3e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696431a81d3b6d6d76a05f3e (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57562 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET /wiki HTTP/1.0" 404 Not Found INFO: 172.20.0.1:50970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37302 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:21:02,168 - lava_callback - INFO - Received callback for node 69620be915f9f48f76d0e267 INFO: 172.20.0.1:33584 - "POST /node/69620be915f9f48f76d0e267 HTTP/1.0" 202 Accepted 2026-01-12 08:21:02,174 - lava_callback - INFO - Starting async job submit for node 69620be915f9f48f76d0e267 2026-01-12 08:21:02,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:21:02,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620be915f9f48f76d0e267 HTTP/1.1" 502 150 2026-01-12 08:21:02,207 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620be915f9f48f76d0e267'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:21:02,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620be915f9f48f76d0e267 2026-01-12 08:21:02,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620be915f9f48f76d0e267 HTTP/1.1" 502 150 2026-01-12 08:21:02,208 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620be915f9f48f76d0e267'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:21:04,208 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620be915f9f48f76d0e267 2026-01-12 08:21:04,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620be915f9f48f76d0e267 HTTP/1.1" 502 150 2026-01-12 08:21:04,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620be915f9f48f76d0e267'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:33600 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:21:08,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620be915f9f48f76d0e267 2026-01-12 08:21:08,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620be915f9f48f76d0e267 HTTP/1.1" 502 150 2026-01-12 08:21:08,211 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620be915f9f48f76d0e267'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:44246 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 08:21:16,211 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620be915f9f48f76d0e267 2026-01-12 08:21:16,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620be915f9f48f76d0e267 HTTP/1.1" 502 150 2026-01-12 08:21:16,214 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620be915f9f48f76d0e267'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53004 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:21:32,215 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620be915f9f48f76d0e267 2026-01-12 08:21:32,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620be915f9f48f76d0e267 HTTP/1.1" 502 150 2026-01-12 08:21:32,216 - lava_callback - ERROR - Error processing callback for node 69620be915f9f48f76d0e267: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620be915f9f48f76d0e267 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620be915f9f48f76d0e267 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620be915f9f48f76d0e267 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48462 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:22:02,208 - lava_callback - INFO - Received callback for node 69620c2015f9f48f76d0e26c INFO: 172.20.0.1:54188 - "POST /node/69620c2015f9f48f76d0e26c HTTP/1.0" 202 Accepted 2026-01-12 08:22:02,214 - lava_callback - INFO - Starting async job submit for node 69620c2015f9f48f76d0e26c 2026-01-12 08:22:02,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:22:02,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c2015f9f48f76d0e26c HTTP/1.1" 502 150 2026-01-12 08:22:02,238 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c2015f9f48f76d0e26c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:22:02,238 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c2015f9f48f76d0e26c 2026-01-12 08:22:02,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c2015f9f48f76d0e26c HTTP/1.1" 502 150 2026-01-12 08:22:02,239 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c2015f9f48f76d0e26c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:22:04,239 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c2015f9f48f76d0e26c 2026-01-12 08:22:04,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c2015f9f48f76d0e26c HTTP/1.1" 502 150 2026-01-12 08:22:04,241 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c2015f9f48f76d0e26c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:54200 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:22:08,241 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c2015f9f48f76d0e26c 2026-01-12 08:22:08,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c2015f9f48f76d0e26c HTTP/1.1" 502 150 2026-01-12 08:22:08,242 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c2015f9f48f76d0e26c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:44516 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 08:22:16,242 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c2015f9f48f76d0e26c 2026-01-12 08:22:16,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c2015f9f48f76d0e26c HTTP/1.1" 502 150 2026-01-12 08:22:16,243 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c2015f9f48f76d0e26c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:44530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60360 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:22:32,244 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c2015f9f48f76d0e26c 2026-01-12 08:22:32,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c2015f9f48f76d0e26c HTTP/1.1" 502 150 2026-01-12 08:22:32,245 - lava_callback - ERROR - Error processing callback for node 69620c2015f9f48f76d0e26c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c2015f9f48f76d0e26c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c2015f9f48f76d0e26c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c2015f9f48f76d0e26c (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35930 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:23:02,293 - lava_callback - INFO - Received callback for node 69620c5615f9f48f76d0e272 INFO: 172.20.0.1:36008 - "POST /node/69620c5615f9f48f76d0e272 HTTP/1.0" 202 Accepted 2026-01-12 08:23:02,298 - lava_callback - INFO - Starting async job submit for node 69620c5615f9f48f76d0e272 2026-01-12 08:23:02,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:23:02,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c5615f9f48f76d0e272 HTTP/1.1" 502 150 2026-01-12 08:23:02,312 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c5615f9f48f76d0e272'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:23:02,312 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c5615f9f48f76d0e272 2026-01-12 08:23:02,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c5615f9f48f76d0e272 HTTP/1.1" 502 150 2026-01-12 08:23:02,313 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c5615f9f48f76d0e272'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:23:04,313 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c5615f9f48f76d0e272 2026-01-12 08:23:04,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c5615f9f48f76d0e272 HTTP/1.1" 502 150 2026-01-12 08:23:04,314 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c5615f9f48f76d0e272'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:36024 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:23:08,315 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c5615f9f48f76d0e272 2026-01-12 08:23:08,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c5615f9f48f76d0e272 HTTP/1.1" 502 150 2026-01-12 08:23:08,316 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c5615f9f48f76d0e272'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:51152 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 08:23:16,316 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c5615f9f48f76d0e272 2026-01-12 08:23:16,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c5615f9f48f76d0e272 HTTP/1.1" 502 150 2026-01-12 08:23:16,317 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c5615f9f48f76d0e272'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:23:32,318 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c5615f9f48f76d0e272 2026-01-12 08:23:32,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c5615f9f48f76d0e272 HTTP/1.1" 502 150 2026-01-12 08:23:32,319 - lava_callback - ERROR - Error processing callback for node 69620c5615f9f48f76d0e272: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c5615f9f48f76d0e272 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c5615f9f48f76d0e272 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c5615f9f48f76d0e272 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:49932 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:23:42,319 - lava_callback - INFO - Received callback for node 69620c8b15f9f48f76d0e277 INFO: 172.20.0.1:45656 - "POST /node/69620c8b15f9f48f76d0e277 HTTP/1.0" 202 Accepted 2026-01-12 08:23:42,334 - lava_callback - INFO - Starting async job submit for node 69620c8b15f9f48f76d0e277 2026-01-12 08:23:42,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:23:42,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c8b15f9f48f76d0e277 HTTP/1.1" 502 150 2026-01-12 08:23:42,354 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c8b15f9f48f76d0e277'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:23:42,354 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c8b15f9f48f76d0e277 2026-01-12 08:23:42,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c8b15f9f48f76d0e277 HTTP/1.1" 502 150 2026-01-12 08:23:42,356 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c8b15f9f48f76d0e277'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:23:44,356 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c8b15f9f48f76d0e277 2026-01-12 08:23:44,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c8b15f9f48f76d0e277 HTTP/1.1" 502 150 2026-01-12 08:23:44,357 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c8b15f9f48f76d0e277'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:45666 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:23:48,358 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c8b15f9f48f76d0e277 2026-01-12 08:23:48,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c8b15f9f48f76d0e277 HTTP/1.1" 502 150 2026-01-12 08:23:48,359 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c8b15f9f48f76d0e277'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:35650 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:23:56,359 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c8b15f9f48f76d0e277 2026-01-12 08:23:56,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c8b15f9f48f76d0e277 HTTP/1.1" 502 150 2026-01-12 08:23:56,360 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620c8b15f9f48f76d0e277'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33218 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:24:12,360 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620c8b15f9f48f76d0e277 2026-01-12 08:24:12,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620c8b15f9f48f76d0e277 HTTP/1.1" 502 150 2026-01-12 08:24:12,362 - lava_callback - ERROR - Error processing callback for node 69620c8b15f9f48f76d0e277: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c8b15f9f48f76d0e277 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c8b15f9f48f76d0e277 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620c8b15f9f48f76d0e277 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:39464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:25:42,375 - lava_callback - INFO - Received callback for node 69620cfa15f9f48f76d0e5ff INFO: 172.20.0.1:35282 - "POST /node/69620cfa15f9f48f76d0e5ff HTTP/1.0" 202 Accepted 2026-01-12 08:25:42,384 - lava_callback - INFO - Starting async job submit for node 69620cfa15f9f48f76d0e5ff 2026-01-12 08:25:42,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:25:42,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620cfa15f9f48f76d0e5ff HTTP/1.1" 502 150 2026-01-12 08:25:42,454 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620cfa15f9f48f76d0e5ff'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:25:42,454 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620cfa15f9f48f76d0e5ff 2026-01-12 08:25:42,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620cfa15f9f48f76d0e5ff HTTP/1.1" 502 150 2026-01-12 08:25:42,456 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620cfa15f9f48f76d0e5ff'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:25:44,456 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620cfa15f9f48f76d0e5ff 2026-01-12 08:25:44,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620cfa15f9f48f76d0e5ff HTTP/1.1" 502 150 2026-01-12 08:25:44,457 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620cfa15f9f48f76d0e5ff'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:35288 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:25:48,457 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620cfa15f9f48f76d0e5ff 2026-01-12 08:25:48,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620cfa15f9f48f76d0e5ff HTTP/1.1" 502 150 2026-01-12 08:25:48,458 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620cfa15f9f48f76d0e5ff'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:32860 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:25:56,459 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620cfa15f9f48f76d0e5ff 2026-01-12 08:25:56,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620cfa15f9f48f76d0e5ff HTTP/1.1" 502 150 2026-01-12 08:25:56,460 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620cfa15f9f48f76d0e5ff'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:26:12,460 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620cfa15f9f48f76d0e5ff 2026-01-12 08:26:12,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620cfa15f9f48f76d0e5ff HTTP/1.1" 502 150 2026-01-12 08:26:12,461 - lava_callback - ERROR - Error processing callback for node 69620cfa15f9f48f76d0e5ff: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620cfa15f9f48f76d0e5ff (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620cfa15f9f48f76d0e5ff (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620cfa15f9f48f76d0e5ff (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:58382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39832 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:26:42,449 - lava_callback - INFO - Received callback for node 69620d3015f9f48f76d0e604 INFO: 172.20.0.1:36690 - "POST /node/69620d3015f9f48f76d0e604 HTTP/1.0" 202 Accepted 2026-01-12 08:26:42,454 - lava_callback - INFO - Starting async job submit for node 69620d3015f9f48f76d0e604 2026-01-12 08:26:42,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:26:42,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620d3015f9f48f76d0e604 HTTP/1.1" 502 150 2026-01-12 08:26:42,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620d3015f9f48f76d0e604'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:26:42,468 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620d3015f9f48f76d0e604 2026-01-12 08:26:42,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620d3015f9f48f76d0e604 HTTP/1.1" 502 150 2026-01-12 08:26:42,468 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620d3015f9f48f76d0e604'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:26:44,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620d3015f9f48f76d0e604 2026-01-12 08:26:44,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620d3015f9f48f76d0e604 HTTP/1.1" 502 150 2026-01-12 08:26:44,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620d3015f9f48f76d0e604'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:36704 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:26:48,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620d3015f9f48f76d0e604 2026-01-12 08:26:48,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620d3015f9f48f76d0e604 HTTP/1.1" 502 150 2026-01-12 08:26:48,474 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620d3015f9f48f76d0e604'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:52716 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:26:56,474 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620d3015f9f48f76d0e604 2026-01-12 08:26:56,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620d3015f9f48f76d0e604 HTTP/1.1" 502 150 2026-01-12 08:26:56,476 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69620d3015f9f48f76d0e604'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57282 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:27:12,476 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69620d3015f9f48f76d0e604 2026-01-12 08:27:12,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69620d3015f9f48f76d0e604 HTTP/1.1" 502 150 2026-01-12 08:27:12,477 - lava_callback - ERROR - Error processing callback for node 69620d3015f9f48f76d0e604: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620d3015f9f48f76d0e604 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620d3015f9f48f76d0e604 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69620d3015f9f48f76d0e604 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:38666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:30:30,687 - lava_callback - INFO - Received callback for node 696438a41d3b6d6d76a05fb1 INFO: 172.20.0.1:60756 - "POST /node/696438a41d3b6d6d76a05fb1 HTTP/1.0" 202 Accepted 2026-01-12 08:30:30,694 - lava_callback - INFO - Starting async job submit for node 696438a41d3b6d6d76a05fb1 2026-01-12 08:30:30,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:30:30,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696438a41d3b6d6d76a05fb1 HTTP/1.1" 502 150 2026-01-12 08:30:30,770 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696438a41d3b6d6d76a05fb1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:30:30,770 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696438a41d3b6d6d76a05fb1 2026-01-12 08:30:30,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696438a41d3b6d6d76a05fb1 HTTP/1.1" 502 150 2026-01-12 08:30:30,771 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696438a41d3b6d6d76a05fb1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:30:32,771 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696438a41d3b6d6d76a05fb1 2026-01-12 08:30:32,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696438a41d3b6d6d76a05fb1 HTTP/1.1" 502 150 2026-01-12 08:30:32,774 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696438a41d3b6d6d76a05fb1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:60762 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:30:36,774 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696438a41d3b6d6d76a05fb1 2026-01-12 08:30:36,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696438a41d3b6d6d76a05fb1 HTTP/1.1" 502 150 2026-01-12 08:30:36,776 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696438a41d3b6d6d76a05fb1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:30:44,776 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696438a41d3b6d6d76a05fb1 2026-01-12 08:30:44,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696438a41d3b6d6d76a05fb1 HTTP/1.1" 502 150 2026-01-12 08:30:44,777 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696438a41d3b6d6d76a05fb1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43088 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:31:00,777 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696438a41d3b6d6d76a05fb1 2026-01-12 08:31:00,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696438a41d3b6d6d76a05fb1 HTTP/1.1" 502 150 2026-01-12 08:31:00,778 - lava_callback - ERROR - Error processing callback for node 696438a41d3b6d6d76a05fb1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696438a41d3b6d6d76a05fb1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696438a41d3b6d6d76a05fb1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696438a41d3b6d6d76a05fb1 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:39086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45104 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:39:43,204 - lava_callback - INFO - Received callback for node 6962103f15f9f48f76d0e63b INFO: 172.20.0.1:60500 - "POST /node/6962103f15f9f48f76d0e63b HTTP/1.0" 202 Accepted 2026-01-12 08:39:43,209 - lava_callback - INFO - Starting async job submit for node 6962103f15f9f48f76d0e63b 2026-01-12 08:39:43,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:39:43,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962103f15f9f48f76d0e63b HTTP/1.1" 502 150 2026-01-12 08:39:43,222 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962103f15f9f48f76d0e63b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:43,222 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962103f15f9f48f76d0e63b 2026-01-12 08:39:43,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962103f15f9f48f76d0e63b HTTP/1.1" 502 150 2026-01-12 08:39:43,223 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962103f15f9f48f76d0e63b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:43,854 - lava_callback - INFO - Received callback for node 6962104115f9f48f76d0e63e 2026-01-12 08:39:43,860 - lava_callback - INFO - Starting async job submit for node 6962104115f9f48f76d0e63e INFO: 172.20.0.1:60502 - "POST /node/6962104115f9f48f76d0e63e HTTP/1.0" 202 Accepted 2026-01-12 08:39:43,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:39:43,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104115f9f48f76d0e63e HTTP/1.1" 502 150 2026-01-12 08:39:43,875 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104115f9f48f76d0e63e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:43,875 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104115f9f48f76d0e63e 2026-01-12 08:39:43,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104115f9f48f76d0e63e HTTP/1.1" 502 150 2026-01-12 08:39:43,876 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104115f9f48f76d0e63e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:44,516 - lava_callback - INFO - Received callback for node 6962104215f9f48f76d0e641 2026-01-12 08:39:44,521 - lava_callback - INFO - Starting async job submit for node 6962104215f9f48f76d0e641 INFO: 172.20.0.1:60514 - "POST /node/6962104215f9f48f76d0e641 HTTP/1.0" 202 Accepted 2026-01-12 08:39:44,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:39:44,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104215f9f48f76d0e641 HTTP/1.1" 502 150 2026-01-12 08:39:44,536 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104215f9f48f76d0e641'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:44,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104215f9f48f76d0e641 2026-01-12 08:39:44,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104215f9f48f76d0e641 HTTP/1.1" 502 150 2026-01-12 08:39:44,537 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104215f9f48f76d0e641'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:45,171 - lava_callback - INFO - Received callback for node 6962104415f9f48f76d0e644 2026-01-12 08:39:45,176 - lava_callback - INFO - Starting async job submit for node 6962104415f9f48f76d0e644 INFO: 172.20.0.1:60530 - "POST /node/6962104415f9f48f76d0e644 HTTP/1.0" 202 Accepted 2026-01-12 08:39:45,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:39:45,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104415f9f48f76d0e644 HTTP/1.1" 502 150 2026-01-12 08:39:45,191 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104415f9f48f76d0e644'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:45,191 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104415f9f48f76d0e644 2026-01-12 08:39:45,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104415f9f48f76d0e644 HTTP/1.1" 502 150 2026-01-12 08:39:45,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104415f9f48f76d0e644'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:45,223 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962103f15f9f48f76d0e63b 2026-01-12 08:39:45,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962103f15f9f48f76d0e63b HTTP/1.1" 502 150 2026-01-12 08:39:45,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962103f15f9f48f76d0e63b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:45,815 - lava_callback - INFO - Received callback for node 6962104715f9f48f76d0e64f 2026-01-12 08:39:45,821 - lava_callback - INFO - Starting async job submit for node 6962104715f9f48f76d0e64f INFO: 172.20.0.1:60534 - "POST /node/6962104715f9f48f76d0e64f HTTP/1.0" 202 Accepted 2026-01-12 08:39:45,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:39:45,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104715f9f48f76d0e64f HTTP/1.1" 502 150 2026-01-12 08:39:45,836 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104715f9f48f76d0e64f'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:45,836 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104715f9f48f76d0e64f 2026-01-12 08:39:45,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104715f9f48f76d0e64f HTTP/1.1" 502 150 2026-01-12 08:39:45,837 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104715f9f48f76d0e64f'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:45,876 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104115f9f48f76d0e63e 2026-01-12 08:39:45,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104115f9f48f76d0e63e HTTP/1.1" 502 150 2026-01-12 08:39:45,877 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104115f9f48f76d0e63e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:46,466 - lava_callback - INFO - Received callback for node 6962104915f9f48f76d0e652 2026-01-12 08:39:46,472 - lava_callback - INFO - Starting async job submit for node 6962104915f9f48f76d0e652 INFO: 172.20.0.1:60544 - "POST /node/6962104915f9f48f76d0e652 HTTP/1.0" 202 Accepted 2026-01-12 08:39:46,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:39:46,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104915f9f48f76d0e652 HTTP/1.1" 502 150 2026-01-12 08:39:46,487 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104915f9f48f76d0e652'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:46,487 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104915f9f48f76d0e652 2026-01-12 08:39:46,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104915f9f48f76d0e652 HTTP/1.1" 502 150 2026-01-12 08:39:46,488 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104915f9f48f76d0e652'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:60552 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:39:46,537 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104215f9f48f76d0e641 2026-01-12 08:39:46,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104215f9f48f76d0e641 HTTP/1.1" 502 150 2026-01-12 08:39:46,538 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104215f9f48f76d0e641'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:47,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104415f9f48f76d0e644 2026-01-12 08:39:47,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104415f9f48f76d0e644 HTTP/1.1" 502 150 2026-01-12 08:39:47,194 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104415f9f48f76d0e644'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:47,837 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104715f9f48f76d0e64f 2026-01-12 08:39:47,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104715f9f48f76d0e64f HTTP/1.1" 502 150 2026-01-12 08:39:47,838 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104715f9f48f76d0e64f'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:48,488 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104915f9f48f76d0e652 2026-01-12 08:39:48,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104915f9f48f76d0e652 HTTP/1.1" 502 150 2026-01-12 08:39:48,489 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104915f9f48f76d0e652'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:49,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962103f15f9f48f76d0e63b 2026-01-12 08:39:49,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962103f15f9f48f76d0e63b HTTP/1.1" 502 150 2026-01-12 08:39:49,227 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962103f15f9f48f76d0e63b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:49,877 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104115f9f48f76d0e63e 2026-01-12 08:39:49,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104115f9f48f76d0e63e HTTP/1.1" 502 150 2026-01-12 08:39:49,878 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104115f9f48f76d0e63e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:50,539 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104215f9f48f76d0e641 2026-01-12 08:39:50,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104215f9f48f76d0e641 HTTP/1.1" 502 150 2026-01-12 08:39:50,540 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104215f9f48f76d0e641'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:51,194 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104415f9f48f76d0e644 2026-01-12 08:39:51,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104415f9f48f76d0e644 HTTP/1.1" 502 150 2026-01-12 08:39:51,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104415f9f48f76d0e644'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:51,839 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104715f9f48f76d0e64f 2026-01-12 08:39:51,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104715f9f48f76d0e64f HTTP/1.1" 502 150 2026-01-12 08:39:51,840 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104715f9f48f76d0e64f'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:55012 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:39:52,489 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104915f9f48f76d0e652 2026-01-12 08:39:52,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104915f9f48f76d0e652 HTTP/1.1" 502 150 2026-01-12 08:39:52,491 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104915f9f48f76d0e652'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:55020 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:39:57,227 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962103f15f9f48f76d0e63b 2026-01-12 08:39:57,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962103f15f9f48f76d0e63b HTTP/1.1" 502 150 2026-01-12 08:39:57,228 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962103f15f9f48f76d0e63b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:57,879 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104115f9f48f76d0e63e 2026-01-12 08:39:57,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104115f9f48f76d0e63e HTTP/1.1" 502 150 2026-01-12 08:39:57,880 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104115f9f48f76d0e63e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:58,541 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104215f9f48f76d0e641 2026-01-12 08:39:58,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104215f9f48f76d0e641 HTTP/1.1" 502 150 2026-01-12 08:39:58,542 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104215f9f48f76d0e641'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:59,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104415f9f48f76d0e644 2026-01-12 08:39:59,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104415f9f48f76d0e644 HTTP/1.1" 502 150 2026-01-12 08:39:59,197 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104415f9f48f76d0e644'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:39:59,840 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104715f9f48f76d0e64f 2026-01-12 08:39:59,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104715f9f48f76d0e64f HTTP/1.1" 502 150 2026-01-12 08:39:59,841 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104715f9f48f76d0e64f'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:40:00,491 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104915f9f48f76d0e652 2026-01-12 08:40:00,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104915f9f48f76d0e652 HTTP/1.1" 502 150 2026-01-12 08:40:00,492 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962104915f9f48f76d0e652'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:46850 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:40:13,228 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962103f15f9f48f76d0e63b 2026-01-12 08:40:13,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962103f15f9f48f76d0e63b HTTP/1.1" 502 150 2026-01-12 08:40:13,230 - lava_callback - ERROR - Error processing callback for node 6962103f15f9f48f76d0e63b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962103f15f9f48f76d0e63b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962103f15f9f48f76d0e63b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962103f15f9f48f76d0e63b (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:40:13,880 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104115f9f48f76d0e63e 2026-01-12 08:40:13,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104115f9f48f76d0e63e HTTP/1.1" 502 150 2026-01-12 08:40:13,881 - lava_callback - ERROR - Error processing callback for node 6962104115f9f48f76d0e63e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104115f9f48f76d0e63e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104115f9f48f76d0e63e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104115f9f48f76d0e63e (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:40:14,542 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104215f9f48f76d0e641 2026-01-12 08:40:14,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104215f9f48f76d0e641 HTTP/1.1" 502 150 2026-01-12 08:40:14,548 - lava_callback - ERROR - Error processing callback for node 6962104215f9f48f76d0e641: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104215f9f48f76d0e641 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104215f9f48f76d0e641 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104215f9f48f76d0e641 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:38218 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 08:40:15,197 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104415f9f48f76d0e644 2026-01-12 08:40:15,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104415f9f48f76d0e644 HTTP/1.1" 502 150 2026-01-12 08:40:15,198 - lava_callback - ERROR - Error processing callback for node 6962104415f9f48f76d0e644: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104415f9f48f76d0e644 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104415f9f48f76d0e644 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104415f9f48f76d0e644 (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:40:15,841 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104715f9f48f76d0e64f 2026-01-12 08:40:15,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104715f9f48f76d0e64f HTTP/1.1" 502 150 2026-01-12 08:40:15,843 - lava_callback - ERROR - Error processing callback for node 6962104715f9f48f76d0e64f: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104715f9f48f76d0e64f (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104715f9f48f76d0e64f (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104715f9f48f76d0e64f (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:40:16,493 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962104915f9f48f76d0e652 2026-01-12 08:40:16,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962104915f9f48f76d0e652 HTTP/1.1" 502 150 2026-01-12 08:40:16,494 - lava_callback - ERROR - Error processing callback for node 6962104915f9f48f76d0e652: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104915f9f48f76d0e652 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104915f9f48f76d0e652 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962104915f9f48f76d0e652 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42966 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:03,825 - lava_callback - INFO - Received callback for node 696212f015f9f48f76d0e7ae INFO: 172.20.0.1:55246 - "POST /node/696212f015f9f48f76d0e7ae HTTP/1.0" 202 Accepted 2026-01-12 08:51:03,830 - lava_callback - INFO - Starting async job submit for node 696212f015f9f48f76d0e7ae 2026-01-12 08:51:03,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:51:03,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f015f9f48f76d0e7ae HTTP/1.1" 502 150 2026-01-12 08:51:03,844 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f015f9f48f76d0e7ae'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:03,844 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f015f9f48f76d0e7ae 2026-01-12 08:51:03,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f015f9f48f76d0e7ae HTTP/1.1" 502 150 2026-01-12 08:51:03,845 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f015f9f48f76d0e7ae'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:04,492 - lava_callback - INFO - Received callback for node 696212f215f9f48f76d0e7b1 INFO: 172.20.0.1:55250 - "POST /node/696212f215f9f48f76d0e7b1 HTTP/1.0" 202 Accepted 2026-01-12 08:51:04,499 - lava_callback - INFO - Starting async job submit for node 696212f215f9f48f76d0e7b1 2026-01-12 08:51:04,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:51:04,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f215f9f48f76d0e7b1 HTTP/1.1" 502 150 2026-01-12 08:51:04,512 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f215f9f48f76d0e7b1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:04,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f215f9f48f76d0e7b1 2026-01-12 08:51:04,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f215f9f48f76d0e7b1 HTTP/1.1" 502 150 2026-01-12 08:51:04,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f215f9f48f76d0e7b1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:05,163 - lava_callback - INFO - Received callback for node 696212f415f9f48f76d0e7b4 INFO: 172.20.0.1:55258 - "POST /node/696212f415f9f48f76d0e7b4 HTTP/1.0" 202 Accepted 2026-01-12 08:51:05,168 - lava_callback - INFO - Starting async job submit for node 696212f415f9f48f76d0e7b4 2026-01-12 08:51:05,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:51:05,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f415f9f48f76d0e7b4 HTTP/1.1" 502 150 2026-01-12 08:51:05,195 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f415f9f48f76d0e7b4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:05,195 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f415f9f48f76d0e7b4 2026-01-12 08:51:05,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f415f9f48f76d0e7b4 HTTP/1.1" 502 150 2026-01-12 08:51:05,196 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f415f9f48f76d0e7b4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:05,845 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f015f9f48f76d0e7ae 2026-01-12 08:51:05,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f015f9f48f76d0e7ae HTTP/1.1" 502 150 2026-01-12 08:51:05,846 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f015f9f48f76d0e7ae'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:06,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f215f9f48f76d0e7b1 2026-01-12 08:51:06,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f215f9f48f76d0e7b1 HTTP/1.1" 502 150 2026-01-12 08:51:06,515 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f215f9f48f76d0e7b1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:55266 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:07,196 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f415f9f48f76d0e7b4 2026-01-12 08:51:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f415f9f48f76d0e7b4 HTTP/1.1" 502 150 2026-01-12 08:51:07,200 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f415f9f48f76d0e7b4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:09,846 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f015f9f48f76d0e7ae 2026-01-12 08:51:09,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f015f9f48f76d0e7ae HTTP/1.1" 502 150 2026-01-12 08:51:09,848 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f015f9f48f76d0e7ae'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:10,515 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f215f9f48f76d0e7b1 2026-01-12 08:51:10,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f215f9f48f76d0e7b1 HTTP/1.1" 502 150 2026-01-12 08:51:10,524 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f215f9f48f76d0e7b1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:11,200 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f415f9f48f76d0e7b4 2026-01-12 08:51:11,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f415f9f48f76d0e7b4 HTTP/1.1" 502 150 2026-01-12 08:51:11,201 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f415f9f48f76d0e7b4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:55416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:17,848 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f015f9f48f76d0e7ae 2026-01-12 08:51:17,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f015f9f48f76d0e7ae HTTP/1.1" 502 150 2026-01-12 08:51:17,849 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f015f9f48f76d0e7ae'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:18,524 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f215f9f48f76d0e7b1 2026-01-12 08:51:18,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f215f9f48f76d0e7b1 HTTP/1.1" 502 150 2026-01-12 08:51:18,525 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f215f9f48f76d0e7b1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:19,201 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f415f9f48f76d0e7b4 2026-01-12 08:51:19,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f415f9f48f76d0e7b4 HTTP/1.1" 502 150 2026-01-12 08:51:19,202 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f415f9f48f76d0e7b4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:23,856 - lava_callback - INFO - Received callback for node 696212f715f9f48f76d0e7bb INFO: 172.20.0.1:45386 - "POST /node/696212f715f9f48f76d0e7bb HTTP/1.0" 202 Accepted 2026-01-12 08:51:23,862 - lava_callback - INFO - Starting async job submit for node 696212f715f9f48f76d0e7bb 2026-01-12 08:51:23,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:51:23,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f715f9f48f76d0e7bb HTTP/1.1" 502 150 2026-01-12 08:51:23,898 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f715f9f48f76d0e7bb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:23,898 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f715f9f48f76d0e7bb 2026-01-12 08:51:23,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f715f9f48f76d0e7bb HTTP/1.1" 502 150 2026-01-12 08:51:23,899 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f715f9f48f76d0e7bb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:24,527 - lava_callback - INFO - Received callback for node 696212fa15f9f48f76d0e7bf INFO: 172.20.0.1:45390 - "POST /node/696212fa15f9f48f76d0e7bf HTTP/1.0" 202 Accepted 2026-01-12 08:51:24,536 - lava_callback - INFO - Starting async job submit for node 696212fa15f9f48f76d0e7bf 2026-01-12 08:51:24,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:51:24,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fa15f9f48f76d0e7bf HTTP/1.1" 502 150 2026-01-12 08:51:24,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fa15f9f48f76d0e7bf'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:24,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fa15f9f48f76d0e7bf 2026-01-12 08:51:24,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fa15f9f48f76d0e7bf HTTP/1.1" 502 150 2026-01-12 08:51:24,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fa15f9f48f76d0e7bf'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:25,176 - lava_callback - INFO - Received callback for node 696212fc15f9f48f76d0e7c2 INFO: 172.20.0.1:45392 - "POST /node/696212fc15f9f48f76d0e7c2 HTTP/1.0" 202 Accepted 2026-01-12 08:51:25,185 - lava_callback - INFO - Starting async job submit for node 696212fc15f9f48f76d0e7c2 2026-01-12 08:51:25,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:51:25,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fc15f9f48f76d0e7c2 HTTP/1.1" 502 150 2026-01-12 08:51:25,209 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fc15f9f48f76d0e7c2'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:25,209 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fc15f9f48f76d0e7c2 2026-01-12 08:51:25,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fc15f9f48f76d0e7c2 HTTP/1.1" 502 150 2026-01-12 08:51:25,210 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fc15f9f48f76d0e7c2'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:25,899 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f715f9f48f76d0e7bb 2026-01-12 08:51:25,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f715f9f48f76d0e7bb HTTP/1.1" 502 150 2026-01-12 08:51:25,900 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f715f9f48f76d0e7bb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:45408 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:26,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fa15f9f48f76d0e7bf 2026-01-12 08:51:26,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fa15f9f48f76d0e7bf HTTP/1.1" 502 150 2026-01-12 08:51:26,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fa15f9f48f76d0e7bf'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:27,210 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fc15f9f48f76d0e7c2 2026-01-12 08:51:27,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fc15f9f48f76d0e7c2 HTTP/1.1" 502 150 2026-01-12 08:51:27,215 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fc15f9f48f76d0e7c2'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:29,901 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f715f9f48f76d0e7bb 2026-01-12 08:51:29,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f715f9f48f76d0e7bb HTTP/1.1" 502 150 2026-01-12 08:51:29,909 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f715f9f48f76d0e7bb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:30,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fa15f9f48f76d0e7bf 2026-01-12 08:51:30,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fa15f9f48f76d0e7bf HTTP/1.1" 502 150 2026-01-12 08:51:30,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fa15f9f48f76d0e7bf'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:31,216 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fc15f9f48f76d0e7c2 2026-01-12 08:51:31,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fc15f9f48f76d0e7c2 HTTP/1.1" 502 150 2026-01-12 08:51:31,217 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fc15f9f48f76d0e7c2'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:33,849 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f015f9f48f76d0e7ae 2026-01-12 08:51:33,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f015f9f48f76d0e7ae HTTP/1.1" 502 150 2026-01-12 08:51:33,851 - lava_callback - ERROR - Error processing callback for node 696212f015f9f48f76d0e7ae: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f015f9f48f76d0e7ae (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f015f9f48f76d0e7ae (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f015f9f48f76d0e7ae (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:51:34,525 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f215f9f48f76d0e7b1 2026-01-12 08:51:34,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f215f9f48f76d0e7b1 HTTP/1.1" 502 150 2026-01-12 08:51:34,527 - lava_callback - ERROR - Error processing callback for node 696212f215f9f48f76d0e7b1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f215f9f48f76d0e7b1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f215f9f48f76d0e7b1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f215f9f48f76d0e7b1 (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:51:35,203 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f415f9f48f76d0e7b4 2026-01-12 08:51:35,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f415f9f48f76d0e7b4 HTTP/1.1" 502 150 2026-01-12 08:51:35,204 - lava_callback - ERROR - Error processing callback for node 696212f415f9f48f76d0e7b4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f415f9f48f76d0e7b4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f415f9f48f76d0e7b4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f415f9f48f76d0e7b4 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:57806 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:37,909 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f715f9f48f76d0e7bb 2026-01-12 08:51:37,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f715f9f48f76d0e7bb HTTP/1.1" 502 150 2026-01-12 08:51:37,910 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212f715f9f48f76d0e7bb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:38,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fa15f9f48f76d0e7bf 2026-01-12 08:51:38,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fa15f9f48f76d0e7bf HTTP/1.1" 502 150 2026-01-12 08:51:38,757 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fa15f9f48f76d0e7bf'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:39,217 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fc15f9f48f76d0e7c2 2026-01-12 08:51:39,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fc15f9f48f76d0e7c2 HTTP/1.1" 502 150 2026-01-12 08:51:39,225 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696212fc15f9f48f76d0e7c2'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:43,861 - lava_callback - INFO - Received callback for node 6962131715f9f48f76d0e7f4 INFO: 172.20.0.1:34484 - "POST /node/6962131715f9f48f76d0e7f4 HTTP/1.0" 202 Accepted 2026-01-12 08:51:43,868 - lava_callback - INFO - Starting async job submit for node 6962131715f9f48f76d0e7f4 2026-01-12 08:51:43,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:51:43,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962131715f9f48f76d0e7f4 HTTP/1.1" 502 150 2026-01-12 08:51:43,884 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962131715f9f48f76d0e7f4'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:43,884 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962131715f9f48f76d0e7f4 2026-01-12 08:51:43,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962131715f9f48f76d0e7f4 HTTP/1.1" 502 150 2026-01-12 08:51:43,885 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962131715f9f48f76d0e7f4'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:51:45,885 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962131715f9f48f76d0e7f4 2026-01-12 08:51:45,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962131715f9f48f76d0e7f4 HTTP/1.1" 502 150 2026-01-12 08:51:45,887 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962131715f9f48f76d0e7f4'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:34498 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:49,887 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962131715f9f48f76d0e7f4 2026-01-12 08:51:49,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962131715f9f48f76d0e7f4 HTTP/1.1" 502 150 2026-01-12 08:51:49,888 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962131715f9f48f76d0e7f4'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:41126 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:53,910 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212f715f9f48f76d0e7bb 2026-01-12 08:51:53,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212f715f9f48f76d0e7bb HTTP/1.1" 502 150 2026-01-12 08:51:53,912 - lava_callback - ERROR - Error processing callback for node 696212f715f9f48f76d0e7bb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f715f9f48f76d0e7bb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f715f9f48f76d0e7bb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212f715f9f48f76d0e7bb (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:51:54,757 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fa15f9f48f76d0e7bf 2026-01-12 08:51:54,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fa15f9f48f76d0e7bf HTTP/1.1" 502 150 2026-01-12 08:51:54,759 - lava_callback - ERROR - Error processing callback for node 696212fa15f9f48f76d0e7bf: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212fa15f9f48f76d0e7bf (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212fa15f9f48f76d0e7bf (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212fa15f9f48f76d0e7bf (Caused by ResponseError('too many 502 error responses')) 2026-01-12 08:51:55,225 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696212fc15f9f48f76d0e7c2 2026-01-12 08:51:55,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696212fc15f9f48f76d0e7c2 HTTP/1.1" 502 150 2026-01-12 08:51:55,228 - lava_callback - ERROR - Error processing callback for node 696212fc15f9f48f76d0e7c2: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212fc15f9f48f76d0e7c2 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212fc15f9f48f76d0e7c2 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696212fc15f9f48f76d0e7c2 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:41132 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:51:57,888 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962131715f9f48f76d0e7f4 2026-01-12 08:51:57,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962131715f9f48f76d0e7f4 HTTP/1.1" 502 150 2026-01-12 08:51:57,890 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962131715f9f48f76d0e7f4'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:60800 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:52:13,890 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962131715f9f48f76d0e7f4 2026-01-12 08:52:13,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962131715f9f48f76d0e7f4 HTTP/1.1" 502 150 2026-01-12 08:52:13,898 - lava_callback - ERROR - Error processing callback for node 6962131715f9f48f76d0e7f4: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962131715f9f48f76d0e7f4 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962131715f9f48f76d0e7f4 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962131715f9f48f76d0e7f4 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:40378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:52:43,917 - lava_callback - INFO - Received callback for node 6962135015f9f48f76d0e827 INFO: 172.20.0.1:52836 - "POST /node/6962135015f9f48f76d0e827 HTTP/1.0" 202 Accepted 2026-01-12 08:52:43,923 - lava_callback - INFO - Starting async job submit for node 6962135015f9f48f76d0e827 2026-01-12 08:52:43,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:52:43,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962135015f9f48f76d0e827 HTTP/1.1" 502 150 2026-01-12 08:52:43,947 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962135015f9f48f76d0e827'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:52:43,947 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962135015f9f48f76d0e827 2026-01-12 08:52:43,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962135015f9f48f76d0e827 HTTP/1.1" 502 150 2026-01-12 08:52:43,948 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962135015f9f48f76d0e827'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:52:45,948 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962135015f9f48f76d0e827 2026-01-12 08:52:45,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962135015f9f48f76d0e827 HTTP/1.1" 502 150 2026-01-12 08:52:45,949 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962135015f9f48f76d0e827'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:52842 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:52:49,950 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962135015f9f48f76d0e827 2026-01-12 08:52:49,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962135015f9f48f76d0e827 HTTP/1.1" 502 150 2026-01-12 08:52:49,951 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962135015f9f48f76d0e827'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42978 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:52:57,951 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962135015f9f48f76d0e827 2026-01-12 08:52:57,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962135015f9f48f76d0e827 HTTP/1.1" 502 150 2026-01-12 08:52:57,952 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962135015f9f48f76d0e827'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:53:13,953 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962135015f9f48f76d0e827 2026-01-12 08:53:13,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962135015f9f48f76d0e827 HTTP/1.1" 502 150 2026-01-12 08:53:13,961 - lava_callback - ERROR - Error processing callback for node 6962135015f9f48f76d0e827: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962135015f9f48f76d0e827 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962135015f9f48f76d0e827 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962135015f9f48f76d0e827 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:48540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55112 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:53:31,534 - lava_callback - INFO - Received callback for node 69643aa61d3b6d6d76a05fb8 INFO: 172.20.0.1:49958 - "POST /node/69643aa61d3b6d6d76a05fb8 HTTP/1.0" 202 Accepted 2026-01-12 08:53:31,541 - lava_callback - INFO - Starting async job submit for node 69643aa61d3b6d6d76a05fb8 2026-01-12 08:53:31,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:53:31,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69643aa61d3b6d6d76a05fb8 HTTP/1.1" 502 150 2026-01-12 08:53:31,594 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69643aa61d3b6d6d76a05fb8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:53:31,594 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69643aa61d3b6d6d76a05fb8 2026-01-12 08:53:31,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69643aa61d3b6d6d76a05fb8 HTTP/1.1" 502 150 2026-01-12 08:53:31,595 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69643aa61d3b6d6d76a05fb8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:53:33,595 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69643aa61d3b6d6d76a05fb8 2026-01-12 08:53:33,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69643aa61d3b6d6d76a05fb8 HTTP/1.1" 502 150 2026-01-12 08:53:33,596 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69643aa61d3b6d6d76a05fb8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:49968 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:53:37,596 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69643aa61d3b6d6d76a05fb8 2026-01-12 08:53:37,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69643aa61d3b6d6d76a05fb8 HTTP/1.1" 502 150 2026-01-12 08:53:37,598 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69643aa61d3b6d6d76a05fb8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:53:43,977 - lava_callback - INFO - Received callback for node 6962138715f9f48f76d0e86c INFO: 172.20.0.1:58324 - "POST /node/6962138715f9f48f76d0e86c HTTP/1.0" 202 Accepted 2026-01-12 08:53:43,987 - lava_callback - INFO - Starting async job submit for node 6962138715f9f48f76d0e86c 2026-01-12 08:53:43,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:53:44,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962138715f9f48f76d0e86c HTTP/1.1" 502 150 2026-01-12 08:53:44,015 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962138715f9f48f76d0e86c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:53:44,015 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962138715f9f48f76d0e86c 2026-01-12 08:53:44,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962138715f9f48f76d0e86c HTTP/1.1" 502 150 2026-01-12 08:53:44,016 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962138715f9f48f76d0e86c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:53:45,598 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69643aa61d3b6d6d76a05fb8 2026-01-12 08:53:45,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69643aa61d3b6d6d76a05fb8 HTTP/1.1" 502 150 2026-01-12 08:53:45,600 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69643aa61d3b6d6d76a05fb8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:53:46,016 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962138715f9f48f76d0e86c 2026-01-12 08:53:46,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962138715f9f48f76d0e86c HTTP/1.1" 502 150 2026-01-12 08:53:46,018 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962138715f9f48f76d0e86c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:58330 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:53:50,018 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962138715f9f48f76d0e86c 2026-01-12 08:53:50,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962138715f9f48f76d0e86c HTTP/1.1" 502 150 2026-01-12 08:53:50,019 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962138715f9f48f76d0e86c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54882 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:53:58,019 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962138715f9f48f76d0e86c 2026-01-12 08:53:58,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962138715f9f48f76d0e86c HTTP/1.1" 502 150 2026-01-12 08:53:58,021 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962138715f9f48f76d0e86c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:54:01,600 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69643aa61d3b6d6d76a05fb8 2026-01-12 08:54:01,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69643aa61d3b6d6d76a05fb8 HTTP/1.1" 502 150 2026-01-12 08:54:01,607 - lava_callback - ERROR - Error processing callback for node 69643aa61d3b6d6d76a05fb8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69643aa61d3b6d6d76a05fb8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69643aa61d3b6d6d76a05fb8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69643aa61d3b6d6d76a05fb8 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:52366 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:54:14,021 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962138715f9f48f76d0e86c 2026-01-12 08:54:14,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962138715f9f48f76d0e86c HTTP/1.1" 502 150 2026-01-12 08:54:14,022 - lava_callback - ERROR - Error processing callback for node 6962138715f9f48f76d0e86c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962138715f9f48f76d0e86c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962138715f9f48f76d0e86c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962138715f9f48f76d0e86c (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:40958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40966 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:54:24,000 - lava_callback - INFO - Received callback for node 696213be15f9f48f76d0e88d INFO: 172.20.0.1:34442 - "POST /node/696213be15f9f48f76d0e88d HTTP/1.0" 202 Accepted 2026-01-12 08:54:24,006 - lava_callback - INFO - Starting async job submit for node 696213be15f9f48f76d0e88d 2026-01-12 08:54:24,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:54:24,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213be15f9f48f76d0e88d HTTP/1.1" 502 150 2026-01-12 08:54:24,022 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213be15f9f48f76d0e88d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:54:24,022 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213be15f9f48f76d0e88d 2026-01-12 08:54:24,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213be15f9f48f76d0e88d HTTP/1.1" 502 150 2026-01-12 08:54:24,023 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213be15f9f48f76d0e88d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:54:26,023 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213be15f9f48f76d0e88d 2026-01-12 08:54:26,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213be15f9f48f76d0e88d HTTP/1.1" 502 150 2026-01-12 08:54:26,024 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213be15f9f48f76d0e88d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:34444 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:54:30,024 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213be15f9f48f76d0e88d 2026-01-12 08:54:30,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213be15f9f48f76d0e88d HTTP/1.1" 502 150 2026-01-12 08:54:30,026 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213be15f9f48f76d0e88d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:57004 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:54:38,026 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213be15f9f48f76d0e88d 2026-01-12 08:54:38,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213be15f9f48f76d0e88d HTTP/1.1" 502 150 2026-01-12 08:54:38,027 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213be15f9f48f76d0e88d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35648 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:54:54,027 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213be15f9f48f76d0e88d 2026-01-12 08:54:54,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213be15f9f48f76d0e88d HTTP/1.1" 502 150 2026-01-12 08:54:54,029 - lava_callback - ERROR - Error processing callback for node 696213be15f9f48f76d0e88d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696213be15f9f48f76d0e88d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696213be15f9f48f76d0e88d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696213be15f9f48f76d0e88d (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34126 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:55:24,098 - lava_callback - INFO - Received callback for node 696213f715f9f48f76d0e8ac INFO: 172.20.0.1:38624 - "POST /node/696213f715f9f48f76d0e8ac HTTP/1.0" 202 Accepted 2026-01-12 08:55:24,104 - lava_callback - INFO - Starting async job submit for node 696213f715f9f48f76d0e8ac 2026-01-12 08:55:24,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:55:24,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213f715f9f48f76d0e8ac HTTP/1.1" 502 150 2026-01-12 08:55:24,117 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213f715f9f48f76d0e8ac'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:55:24,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213f715f9f48f76d0e8ac 2026-01-12 08:55:24,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213f715f9f48f76d0e8ac HTTP/1.1" 502 150 2026-01-12 08:55:24,118 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213f715f9f48f76d0e8ac'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:55:26,118 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213f715f9f48f76d0e8ac 2026-01-12 08:55:26,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213f715f9f48f76d0e8ac HTTP/1.1" 502 150 2026-01-12 08:55:26,120 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213f715f9f48f76d0e8ac'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:38628 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:55:30,120 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213f715f9f48f76d0e8ac 2026-01-12 08:55:30,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213f715f9f48f76d0e8ac HTTP/1.1" 502 150 2026-01-12 08:55:30,121 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213f715f9f48f76d0e8ac'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:55:38,121 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213f715f9f48f76d0e8ac 2026-01-12 08:55:38,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213f715f9f48f76d0e8ac HTTP/1.1" 502 150 2026-01-12 08:55:38,123 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696213f715f9f48f76d0e8ac'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36166 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:55:54,123 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696213f715f9f48f76d0e8ac 2026-01-12 08:55:54,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696213f715f9f48f76d0e8ac HTTP/1.1" 502 150 2026-01-12 08:55:54,124 - lava_callback - ERROR - Error processing callback for node 696213f715f9f48f76d0e8ac: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696213f715f9f48f76d0e8ac (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696213f715f9f48f76d0e8ac (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696213f715f9f48f76d0e8ac (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:56:24,156 - lava_callback - INFO - Received callback for node 6962142d15f9f48f76d0e8bc INFO: 172.20.0.1:60154 - "POST /node/6962142d15f9f48f76d0e8bc HTTP/1.0" 202 Accepted 2026-01-12 08:56:24,166 - lava_callback - INFO - Starting async job submit for node 6962142d15f9f48f76d0e8bc 2026-01-12 08:56:24,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 08:56:24,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962142d15f9f48f76d0e8bc HTTP/1.1" 502 150 2026-01-12 08:56:24,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962142d15f9f48f76d0e8bc'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:56:24,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962142d15f9f48f76d0e8bc 2026-01-12 08:56:24,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962142d15f9f48f76d0e8bc HTTP/1.1" 502 150 2026-01-12 08:56:24,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962142d15f9f48f76d0e8bc'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 08:56:26,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962142d15f9f48f76d0e8bc 2026-01-12 08:56:26,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962142d15f9f48f76d0e8bc HTTP/1.1" 502 150 2026-01-12 08:56:26,190 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962142d15f9f48f76d0e8bc'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:60170 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:56:30,190 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962142d15f9f48f76d0e8bc 2026-01-12 08:56:30,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962142d15f9f48f76d0e8bc HTTP/1.1" 502 150 2026-01-12 08:56:30,192 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962142d15f9f48f76d0e8bc'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:56:38,192 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962142d15f9f48f76d0e8bc 2026-01-12 08:56:38,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962142d15f9f48f76d0e8bc HTTP/1.1" 502 150 2026-01-12 08:56:38,193 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962142d15f9f48f76d0e8bc'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:46322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42608 - "GET / HTTP/1.0" 200 OK 2026-01-12 08:56:54,193 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962142d15f9f48f76d0e8bc 2026-01-12 08:56:54,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962142d15f9f48f76d0e8bc HTTP/1.1" 502 150 2026-01-12 08:56:54,195 - lava_callback - ERROR - Error processing callback for node 6962142d15f9f48f76d0e8bc: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962142d15f9f48f76d0e8bc (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962142d15f9f48f76d0e8bc (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962142d15f9f48f76d0e8bc (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:11:25,015 - lava_callback - INFO - Received callback for node 696217b115f9f48f76d0e938 INFO: 172.20.0.1:45054 - "POST /node/696217b115f9f48f76d0e938 HTTP/1.0" 202 Accepted 2026-01-12 09:11:25,021 - lava_callback - INFO - Starting async job submit for node 696217b115f9f48f76d0e938 2026-01-12 09:11:25,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:11:25,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696217b115f9f48f76d0e938 HTTP/1.1" 502 150 2026-01-12 09:11:25,033 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696217b115f9f48f76d0e938'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:11:25,033 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696217b115f9f48f76d0e938 2026-01-12 09:11:25,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696217b115f9f48f76d0e938 HTTP/1.1" 502 150 2026-01-12 09:11:25,034 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696217b115f9f48f76d0e938'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:45058 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:11:27,034 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696217b115f9f48f76d0e938 2026-01-12 09:11:27,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696217b115f9f48f76d0e938 HTTP/1.1" 502 150 2026-01-12 09:11:27,036 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696217b115f9f48f76d0e938'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:11:31,036 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696217b115f9f48f76d0e938 2026-01-12 09:11:31,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696217b115f9f48f76d0e938 HTTP/1.1" 502 150 2026-01-12 09:11:31,037 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696217b115f9f48f76d0e938'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:41532 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:11:39,037 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696217b115f9f48f76d0e938 2026-01-12 09:11:39,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696217b115f9f48f76d0e938 HTTP/1.1" 502 150 2026-01-12 09:11:39,038 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696217b115f9f48f76d0e938'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46748 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:11:55,038 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696217b115f9f48f76d0e938 2026-01-12 09:11:55,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696217b115f9f48f76d0e938 HTTP/1.1" 502 150 2026-01-12 09:11:55,040 - lava_callback - ERROR - Error processing callback for node 696217b115f9f48f76d0e938: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696217b115f9f48f76d0e938 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696217b115f9f48f76d0e938 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696217b115f9f48f76d0e938 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43540 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 09:13:15,567 - lava_callback - INFO - Received callback for node 6964487c427831b949bfec13 INFO: 172.20.0.1:43552 - "POST /node/6964487c427831b949bfec13 HTTP/1.0" 202 Accepted 2026-01-12 09:13:15,617 - lava_callback - INFO - Starting async job submit for node 6964487c427831b949bfec13 2026-01-12 09:13:15,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:13:15,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964487c427831b949bfec13 HTTP/1.1" 502 150 2026-01-12 09:13:15,654 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964487c427831b949bfec13'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:13:15,654 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964487c427831b949bfec13 2026-01-12 09:13:15,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964487c427831b949bfec13 HTTP/1.1" 502 150 2026-01-12 09:13:15,655 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964487c427831b949bfec13'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:43556 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:13:17,655 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964487c427831b949bfec13 2026-01-12 09:13:17,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964487c427831b949bfec13 HTTP/1.1" 502 150 2026-01-12 09:13:17,657 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964487c427831b949bfec13'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:13:21,657 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964487c427831b949bfec13 2026-01-12 09:13:21,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964487c427831b949bfec13 HTTP/1.1" 502 150 2026-01-12 09:13:21,658 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964487c427831b949bfec13'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:13:29,658 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964487c427831b949bfec13 2026-01-12 09:13:29,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964487c427831b949bfec13 HTTP/1.1" 502 150 2026-01-12 09:13:29,659 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964487c427831b949bfec13'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:54818 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:13:45,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964487c427831b949bfec13 2026-01-12 09:13:45,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964487c427831b949bfec13 HTTP/1.1" 502 150 2026-01-12 09:13:45,661 - lava_callback - ERROR - Error processing callback for node 6964487c427831b949bfec13: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6964487c427831b949bfec13 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6964487c427831b949bfec13 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6964487c427831b949bfec13 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44616 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:14:52,216 - lava_callback - INFO - Received callback for node 696440a6427831b949bfeb52 INFO: 172.20.0.1:47824 - "POST /node/696440a6427831b949bfeb52 HTTP/1.0" 202 Accepted 2026-01-12 09:14:52,222 - lava_callback - INFO - Starting async job submit for node 696440a6427831b949bfeb52 2026-01-12 09:14:52,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:14:52,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a6427831b949bfeb52 HTTP/1.1" 502 150 2026-01-12 09:14:52,304 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a6427831b949bfeb52'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:14:52,304 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a6427831b949bfeb52 2026-01-12 09:14:52,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a6427831b949bfeb52 HTTP/1.1" 502 150 2026-01-12 09:14:52,305 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a6427831b949bfeb52'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:47836 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:14:54,305 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a6427831b949bfeb52 2026-01-12 09:14:54,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a6427831b949bfeb52 HTTP/1.1" 502 150 2026-01-12 09:14:54,306 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a6427831b949bfeb52'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:47848 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:14:58,306 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a6427831b949bfeb52 2026-01-12 09:14:58,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a6427831b949bfeb52 HTTP/1.1" 502 150 2026-01-12 09:14:58,307 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a6427831b949bfeb52'): Retry(total=1, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:15:06,308 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a6427831b949bfeb52 2026-01-12 09:15:06,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a6427831b949bfeb52 HTTP/1.1" 502 150 2026-01-12 09:15:06,309 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a6427831b949bfeb52'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51012 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:15:22,309 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a6427831b949bfeb52 2026-01-12 09:15:22,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a6427831b949bfeb52 HTTP/1.1" 502 150 2026-01-12 09:15:22,311 - lava_callback - ERROR - Error processing callback for node 696440a6427831b949bfeb52: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696440a6427831b949bfeb52 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696440a6427831b949bfeb52 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696440a6427831b949bfeb52 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53848 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:18:25,420 - lava_callback - INFO - Received callback for node 6962195315f9f48f76d0e96e INFO: 172.20.0.1:36404 - "POST /node/6962195315f9f48f76d0e96e HTTP/1.0" 202 Accepted 2026-01-12 09:18:25,426 - lava_callback - INFO - Starting async job submit for node 6962195315f9f48f76d0e96e 2026-01-12 09:18:25,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:18:25,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962195315f9f48f76d0e96e HTTP/1.1" 502 150 2026-01-12 09:18:25,466 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962195315f9f48f76d0e96e'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:18:25,466 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962195315f9f48f76d0e96e 2026-01-12 09:18:25,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962195315f9f48f76d0e96e HTTP/1.1" 502 150 2026-01-12 09:18:25,467 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962195315f9f48f76d0e96e'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:18:27,467 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962195315f9f48f76d0e96e 2026-01-12 09:18:27,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962195315f9f48f76d0e96e HTTP/1.1" 502 150 2026-01-12 09:18:27,469 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962195315f9f48f76d0e96e'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:18:31,469 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962195315f9f48f76d0e96e 2026-01-12 09:18:31,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962195315f9f48f76d0e96e HTTP/1.1" 502 150 2026-01-12 09:18:31,470 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962195315f9f48f76d0e96e'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:58632 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:18:39,470 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962195315f9f48f76d0e96e 2026-01-12 09:18:39,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962195315f9f48f76d0e96e HTTP/1.1" 502 150 2026-01-12 09:18:39,472 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6962195315f9f48f76d0e96e'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49766 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:18:55,472 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6962195315f9f48f76d0e96e 2026-01-12 09:18:55,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6962195315f9f48f76d0e96e HTTP/1.1" 502 150 2026-01-12 09:18:55,473 - lava_callback - ERROR - Error processing callback for node 6962195315f9f48f76d0e96e: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962195315f9f48f76d0e96e (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962195315f9f48f76d0e96e (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6962195315f9f48f76d0e96e (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:19:45,479 - lava_callback - INFO - Received callback for node 696219a315f9f48f76d0e973 INFO: 172.20.0.1:53758 - "POST /node/696219a315f9f48f76d0e973 HTTP/1.0" 202 Accepted 2026-01-12 09:19:45,488 - lava_callback - INFO - Starting async job submit for node 696219a315f9f48f76d0e973 2026-01-12 09:19:45,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:19:45,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219a315f9f48f76d0e973 HTTP/1.1" 502 150 2026-01-12 09:19:45,511 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219a315f9f48f76d0e973'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:19:45,512 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219a315f9f48f76d0e973 2026-01-12 09:19:45,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219a315f9f48f76d0e973 HTTP/1.1" 502 150 2026-01-12 09:19:45,513 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219a315f9f48f76d0e973'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:53772 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:19:47,513 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219a315f9f48f76d0e973 2026-01-12 09:19:47,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219a315f9f48f76d0e973 HTTP/1.1" 502 150 2026-01-12 09:19:47,514 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219a315f9f48f76d0e973'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:19:51,514 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219a315f9f48f76d0e973 2026-01-12 09:19:51,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219a315f9f48f76d0e973 HTTP/1.1" 502 150 2026-01-12 09:19:51,516 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219a315f9f48f76d0e973'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:19:59,516 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219a315f9f48f76d0e973 2026-01-12 09:19:59,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219a315f9f48f76d0e973 HTTP/1.1" 502 150 2026-01-12 09:19:59,517 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219a315f9f48f76d0e973'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:36294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41028 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 09:20:15,518 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219a315f9f48f76d0e973 2026-01-12 09:20:15,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219a315f9f48f76d0e973 HTTP/1.1" 502 150 2026-01-12 09:20:15,519 - lava_callback - ERROR - Error processing callback for node 696219a315f9f48f76d0e973: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696219a315f9f48f76d0e973 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696219a315f9f48f76d0e973 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696219a315f9f48f76d0e973 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41772 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:21:05,552 - lava_callback - INFO - Received callback for node 696219f615f9f48f76d0e97b INFO: 172.20.0.1:50388 - "POST /node/696219f615f9f48f76d0e97b HTTP/1.0" 202 Accepted 2026-01-12 09:21:05,561 - lava_callback - INFO - Starting async job submit for node 696219f615f9f48f76d0e97b 2026-01-12 09:21:05,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:21:05,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219f615f9f48f76d0e97b HTTP/1.1" 502 150 2026-01-12 09:21:05,583 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219f615f9f48f76d0e97b'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:21:05,583 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219f615f9f48f76d0e97b 2026-01-12 09:21:05,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219f615f9f48f76d0e97b HTTP/1.1" 502 150 2026-01-12 09:21:05,585 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219f615f9f48f76d0e97b'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:50402 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:21:07,587 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219f615f9f48f76d0e97b 2026-01-12 09:21:07,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219f615f9f48f76d0e97b HTTP/1.1" 502 150 2026-01-12 09:21:07,588 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219f615f9f48f76d0e97b'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:48912 - "GET /node/693de36f77ecffe173fa0bbc%5Cn HTTP/1.0" 405 Method Not Allowed 2026-01-12 09:21:11,589 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219f615f9f48f76d0e97b 2026-01-12 09:21:11,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219f615f9f48f76d0e97b HTTP/1.1" 502 150 2026-01-12 09:21:11,590 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219f615f9f48f76d0e97b'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:48928 - "GET /node/693de36f77ecffe173fa0bbc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48954 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:21:19,590 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219f615f9f48f76d0e97b 2026-01-12 09:21:19,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219f615f9f48f76d0e97b HTTP/1.1" 502 150 2026-01-12 09:21:19,591 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696219f615f9f48f76d0e97b'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:48850 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:21:35,591 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696219f615f9f48f76d0e97b 2026-01-12 09:21:35,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696219f615f9f48f76d0e97b HTTP/1.1" 502 150 2026-01-12 09:21:35,593 - lava_callback - ERROR - Error processing callback for node 696219f615f9f48f76d0e97b: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696219f615f9f48f76d0e97b (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696219f615f9f48f76d0e97b (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696219f615f9f48f76d0e97b (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36660 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:22:25,614 - lava_callback - INFO - Received callback for node 69621a4915f9f48f76d0e984 INFO: 172.20.0.1:48212 - "POST /node/69621a4915f9f48f76d0e984 HTTP/1.0" 202 Accepted 2026-01-12 09:22:25,620 - lava_callback - INFO - Starting async job submit for node 69621a4915f9f48f76d0e984 2026-01-12 09:22:25,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:22:25,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a4915f9f48f76d0e984 HTTP/1.1" 502 150 2026-01-12 09:22:25,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a4915f9f48f76d0e984'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:22:25,660 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a4915f9f48f76d0e984 2026-01-12 09:22:25,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a4915f9f48f76d0e984 HTTP/1.1" 502 150 2026-01-12 09:22:25,660 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a4915f9f48f76d0e984'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:48224 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:22:27,661 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a4915f9f48f76d0e984 2026-01-12 09:22:27,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a4915f9f48f76d0e984 HTTP/1.1" 502 150 2026-01-12 09:22:27,662 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a4915f9f48f76d0e984'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:22:31,662 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a4915f9f48f76d0e984 2026-01-12 09:22:31,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a4915f9f48f76d0e984 HTTP/1.1" 502 150 2026-01-12 09:22:31,663 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a4915f9f48f76d0e984'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:36072 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:22:39,664 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a4915f9f48f76d0e984 2026-01-12 09:22:39,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a4915f9f48f76d0e984 HTTP/1.1" 502 150 2026-01-12 09:22:39,665 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a4915f9f48f76d0e984'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:22:55,665 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a4915f9f48f76d0e984 2026-01-12 09:22:55,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a4915f9f48f76d0e984 HTTP/1.1" 502 150 2026-01-12 09:22:55,666 - lava_callback - ERROR - Error processing callback for node 69621a4915f9f48f76d0e984: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621a4915f9f48f76d0e984 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621a4915f9f48f76d0e984 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621a4915f9f48f76d0e984 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36578 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:23:45,677 - lava_callback - INFO - Received callback for node 69621a9b15f9f48f76d0e988 INFO: 172.20.0.1:35872 - "POST /node/69621a9b15f9f48f76d0e988 HTTP/1.0" 202 Accepted 2026-01-12 09:23:45,683 - lava_callback - INFO - Starting async job submit for node 69621a9b15f9f48f76d0e988 2026-01-12 09:23:45,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:23:45,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a9b15f9f48f76d0e988 HTTP/1.1" 502 150 2026-01-12 09:23:45,696 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a9b15f9f48f76d0e988'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:23:45,696 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a9b15f9f48f76d0e988 2026-01-12 09:23:45,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a9b15f9f48f76d0e988 HTTP/1.1" 502 150 2026-01-12 09:23:45,697 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a9b15f9f48f76d0e988'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:35884 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:23:47,697 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a9b15f9f48f76d0e988 2026-01-12 09:23:47,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a9b15f9f48f76d0e988 HTTP/1.1" 502 150 2026-01-12 09:23:47,698 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a9b15f9f48f76d0e988'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:23:51,698 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a9b15f9f48f76d0e988 2026-01-12 09:23:51,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a9b15f9f48f76d0e988 HTTP/1.1" 502 150 2026-01-12 09:23:51,700 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a9b15f9f48f76d0e988'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:23:59,700 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a9b15f9f48f76d0e988 2026-01-12 09:23:59,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a9b15f9f48f76d0e988 HTTP/1.1" 502 150 2026-01-12 09:23:59,701 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621a9b15f9f48f76d0e988'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:50248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55708 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 09:24:15,702 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621a9b15f9f48f76d0e988 2026-01-12 09:24:15,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621a9b15f9f48f76d0e988 HTTP/1.1" 502 150 2026-01-12 09:24:15,703 - lava_callback - ERROR - Error processing callback for node 69621a9b15f9f48f76d0e988: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621a9b15f9f48f76d0e988 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621a9b15f9f48f76d0e988 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621a9b15f9f48f76d0e988 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54124 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:25:05,768 - lava_callback - INFO - Received callback for node 69621aef15f9f48f76d0e98c INFO: 172.20.0.1:55216 - "POST /node/69621aef15f9f48f76d0e98c HTTP/1.0" 202 Accepted 2026-01-12 09:25:05,774 - lava_callback - INFO - Starting async job submit for node 69621aef15f9f48f76d0e98c 2026-01-12 09:25:05,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:25:05,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621aef15f9f48f76d0e98c HTTP/1.1" 502 150 2026-01-12 09:25:05,787 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621aef15f9f48f76d0e98c'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:25:05,787 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621aef15f9f48f76d0e98c 2026-01-12 09:25:05,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621aef15f9f48f76d0e98c HTTP/1.1" 502 150 2026-01-12 09:25:05,788 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621aef15f9f48f76d0e98c'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:25:07,788 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621aef15f9f48f76d0e98c 2026-01-12 09:25:07,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621aef15f9f48f76d0e98c HTTP/1.1" 502 150 2026-01-12 09:25:07,789 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621aef15f9f48f76d0e98c'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:25:11,790 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621aef15f9f48f76d0e98c 2026-01-12 09:25:11,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621aef15f9f48f76d0e98c HTTP/1.1" 502 150 2026-01-12 09:25:11,791 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621aef15f9f48f76d0e98c'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:38250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38260 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:25:19,791 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621aef15f9f48f76d0e98c 2026-01-12 09:25:19,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621aef15f9f48f76d0e98c HTTP/1.1" 502 150 2026-01-12 09:25:19,792 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621aef15f9f48f76d0e98c'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:44654 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:25:35,793 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621aef15f9f48f76d0e98c 2026-01-12 09:25:35,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621aef15f9f48f76d0e98c HTTP/1.1" 502 150 2026-01-12 09:25:35,794 - lava_callback - ERROR - Error processing callback for node 69621aef15f9f48f76d0e98c: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621aef15f9f48f76d0e98c (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621aef15f9f48f76d0e98c (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621aef15f9f48f76d0e98c (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57882 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:26:45,832 - lava_callback - INFO - Received callback for node 69621b4215f9f48f76d0e995 INFO: 172.20.0.1:53630 - "POST /node/69621b4215f9f48f76d0e995 HTTP/1.0" 202 Accepted 2026-01-12 09:26:45,837 - lava_callback - INFO - Starting async job submit for node 69621b4215f9f48f76d0e995 2026-01-12 09:26:45,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:26:45,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b4215f9f48f76d0e995 HTTP/1.1" 502 150 2026-01-12 09:26:45,850 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b4215f9f48f76d0e995'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:26:45,850 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b4215f9f48f76d0e995 2026-01-12 09:26:45,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b4215f9f48f76d0e995 HTTP/1.1" 502 150 2026-01-12 09:26:45,851 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b4215f9f48f76d0e995'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:53646 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:26:47,851 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b4215f9f48f76d0e995 2026-01-12 09:26:47,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b4215f9f48f76d0e995 HTTP/1.1" 502 150 2026-01-12 09:26:47,852 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b4215f9f48f76d0e995'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:26:51,853 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b4215f9f48f76d0e995 2026-01-12 09:26:51,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b4215f9f48f76d0e995 HTTP/1.1" 502 150 2026-01-12 09:26:51,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b4215f9f48f76d0e995'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:39880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39894 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:26:59,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b4215f9f48f76d0e995 2026-01-12 09:26:59,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b4215f9f48f76d0e995 HTTP/1.1" 502 150 2026-01-12 09:26:59,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b4215f9f48f76d0e995'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39650 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 09:27:15,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b4215f9f48f76d0e995 2026-01-12 09:27:15,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b4215f9f48f76d0e995 HTTP/1.1" 502 150 2026-01-12 09:27:15,857 - lava_callback - ERROR - Error processing callback for node 69621b4215f9f48f76d0e995: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621b4215f9f48f76d0e995 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621b4215f9f48f76d0e995 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621b4215f9f48f76d0e995 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:39662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51676 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:28:05,892 - lava_callback - INFO - Received callback for node 69621b9415f9f48f76d0e99d INFO: 172.20.0.1:52718 - "POST /node/69621b9415f9f48f76d0e99d HTTP/1.0" 202 Accepted 2026-01-12 09:28:05,898 - lava_callback - INFO - Starting async job submit for node 69621b9415f9f48f76d0e99d 2026-01-12 09:28:05,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:28:05,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b9415f9f48f76d0e99d HTTP/1.1" 502 150 2026-01-12 09:28:05,916 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b9415f9f48f76d0e99d'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:28:05,916 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b9415f9f48f76d0e99d 2026-01-12 09:28:05,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b9415f9f48f76d0e99d HTTP/1.1" 502 150 2026-01-12 09:28:05,917 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b9415f9f48f76d0e99d'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:52730 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:28:07,917 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b9415f9f48f76d0e99d 2026-01-12 09:28:07,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b9415f9f48f76d0e99d HTTP/1.1" 502 150 2026-01-12 09:28:07,919 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b9415f9f48f76d0e99d'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:28:11,919 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b9415f9f48f76d0e99d 2026-01-12 09:28:11,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b9415f9f48f76d0e99d HTTP/1.1" 502 150 2026-01-12 09:28:11,920 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b9415f9f48f76d0e99d'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:45224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45238 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:28:19,920 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b9415f9f48f76d0e99d 2026-01-12 09:28:19,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b9415f9f48f76d0e99d HTTP/1.1" 502 150 2026-01-12 09:28:19,922 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621b9415f9f48f76d0e99d'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:45654 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:28:35,922 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621b9415f9f48f76d0e99d 2026-01-12 09:28:35,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621b9415f9f48f76d0e99d HTTP/1.1" 502 150 2026-01-12 09:28:35,924 - lava_callback - ERROR - Error processing callback for node 69621b9415f9f48f76d0e99d: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621b9415f9f48f76d0e99d (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621b9415f9f48f76d0e99d (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621b9415f9f48f76d0e99d (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:59882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36150 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:29:26,004 - lava_callback - INFO - Received callback for node 69621be415f9f48f76d0e9a1 INFO: 172.20.0.1:35024 - "POST /node/69621be415f9f48f76d0e9a1 HTTP/1.0" 202 Accepted 2026-01-12 09:29:26,015 - lava_callback - INFO - Starting async job submit for node 69621be415f9f48f76d0e9a1 2026-01-12 09:29:26,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:29:26,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621be415f9f48f76d0e9a1 HTTP/1.1" 502 150 2026-01-12 09:29:26,045 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621be415f9f48f76d0e9a1'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:29:26,045 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621be415f9f48f76d0e9a1 2026-01-12 09:29:26,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621be415f9f48f76d0e9a1 HTTP/1.1" 502 150 2026-01-12 09:29:26,046 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621be415f9f48f76d0e9a1'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:35028 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:29:28,046 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621be415f9f48f76d0e9a1 2026-01-12 09:29:28,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621be415f9f48f76d0e9a1 HTTP/1.1" 502 150 2026-01-12 09:29:28,048 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621be415f9f48f76d0e9a1'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:29:32,049 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621be415f9f48f76d0e9a1 2026-01-12 09:29:32,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621be415f9f48f76d0e9a1 HTTP/1.1" 502 150 2026-01-12 09:29:32,050 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621be415f9f48f76d0e9a1'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:58218 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:29:40,050 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621be415f9f48f76d0e9a1 2026-01-12 09:29:40,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621be415f9f48f76d0e9a1 HTTP/1.1" 502 150 2026-01-12 09:29:40,051 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621be415f9f48f76d0e9a1'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:37924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48234 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:29:56,052 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621be415f9f48f76d0e9a1 2026-01-12 09:29:56,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621be415f9f48f76d0e9a1 HTTP/1.1" 502 150 2026-01-12 09:29:56,053 - lava_callback - ERROR - Error processing callback for node 69621be415f9f48f76d0e9a1: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621be415f9f48f76d0e9a1 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621be415f9f48f76d0e9a1 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621be415f9f48f76d0e9a1 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36306 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:30:46,067 - lava_callback - INFO - Received callback for node 69621c3815f9f48f76d0e9a8 INFO: 172.20.0.1:46608 - "POST /node/69621c3815f9f48f76d0e9a8 HTTP/1.0" 202 Accepted 2026-01-12 09:30:46,073 - lava_callback - INFO - Starting async job submit for node 69621c3815f9f48f76d0e9a8 2026-01-12 09:30:46,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:30:46,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c3815f9f48f76d0e9a8 HTTP/1.1" 502 150 2026-01-12 09:30:46,086 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c3815f9f48f76d0e9a8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:30:46,086 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c3815f9f48f76d0e9a8 2026-01-12 09:30:46,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c3815f9f48f76d0e9a8 HTTP/1.1" 502 150 2026-01-12 09:30:46,087 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c3815f9f48f76d0e9a8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:46616 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:30:48,087 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c3815f9f48f76d0e9a8 2026-01-12 09:30:48,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c3815f9f48f76d0e9a8 HTTP/1.1" 502 150 2026-01-12 09:30:48,088 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c3815f9f48f76d0e9a8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:30:52,088 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c3815f9f48f76d0e9a8 2026-01-12 09:30:52,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c3815f9f48f76d0e9a8 HTTP/1.1" 502 150 2026-01-12 09:30:52,090 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c3815f9f48f76d0e9a8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55524 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:31:00,090 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c3815f9f48f76d0e9a8 2026-01-12 09:31:00,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c3815f9f48f76d0e9a8 HTTP/1.1" 502 150 2026-01-12 09:31:00,091 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c3815f9f48f76d0e9a8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:41940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53960 - "GET /api/metrics HTTP/1.0" 200 OK 2026-01-12 09:31:16,091 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c3815f9f48f76d0e9a8 2026-01-12 09:31:16,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c3815f9f48f76d0e9a8 HTTP/1.1" 502 150 2026-01-12 09:31:16,093 - lava_callback - ERROR - Error processing callback for node 69621c3815f9f48f76d0e9a8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621c3815f9f48f76d0e9a8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621c3815f9f48f76d0e9a8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621c3815f9f48f76d0e9a8 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39790 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:32:06,125 - lava_callback - INFO - Received callback for node 69621c8b15f9f48f76d0e9b0 INFO: 172.20.0.1:34606 - "POST /node/69621c8b15f9f48f76d0e9b0 HTTP/1.0" 202 Accepted 2026-01-12 09:32:06,131 - lava_callback - INFO - Starting async job submit for node 69621c8b15f9f48f76d0e9b0 2026-01-12 09:32:06,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:32:06,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c8b15f9f48f76d0e9b0 HTTP/1.1" 502 150 2026-01-12 09:32:06,166 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c8b15f9f48f76d0e9b0'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:32:06,166 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c8b15f9f48f76d0e9b0 2026-01-12 09:32:06,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c8b15f9f48f76d0e9b0 HTTP/1.1" 502 150 2026-01-12 09:32:06,167 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c8b15f9f48f76d0e9b0'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:34620 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:32:08,167 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c8b15f9f48f76d0e9b0 2026-01-12 09:32:08,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c8b15f9f48f76d0e9b0 HTTP/1.1" 502 150 2026-01-12 09:32:08,169 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c8b15f9f48f76d0e9b0'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:32:12,169 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c8b15f9f48f76d0e9b0 2026-01-12 09:32:12,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c8b15f9f48f76d0e9b0 HTTP/1.1" 502 150 2026-01-12 09:32:12,170 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c8b15f9f48f76d0e9b0'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:36758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:32:20,170 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c8b15f9f48f76d0e9b0 2026-01-12 09:32:20,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c8b15f9f48f76d0e9b0 HTTP/1.1" 502 150 2026-01-12 09:32:20,172 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621c8b15f9f48f76d0e9b0'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:43630 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:32:36,172 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621c8b15f9f48f76d0e9b0 2026-01-12 09:32:36,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621c8b15f9f48f76d0e9b0 HTTP/1.1" 502 150 2026-01-12 09:32:36,174 - lava_callback - ERROR - Error processing callback for node 69621c8b15f9f48f76d0e9b0: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621c8b15f9f48f76d0e9b0 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621c8b15f9f48f76d0e9b0 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621c8b15f9f48f76d0e9b0 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:37:53,104 - lava_callback - INFO - Received callback for node 696440a8427831b949bfeb55 INFO: 172.20.0.1:50868 - "POST /node/696440a8427831b949bfeb55 HTTP/1.0" 202 Accepted 2026-01-12 09:37:53,110 - lava_callback - INFO - Starting async job submit for node 696440a8427831b949bfeb55 2026-01-12 09:37:53,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:37:53,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a8427831b949bfeb55 HTTP/1.1" 502 150 2026-01-12 09:37:53,183 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a8427831b949bfeb55'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:37:53,183 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a8427831b949bfeb55 2026-01-12 09:37:53,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a8427831b949bfeb55 HTTP/1.1" 502 150 2026-01-12 09:37:53,184 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a8427831b949bfeb55'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:37:55,184 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a8427831b949bfeb55 2026-01-12 09:37:55,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a8427831b949bfeb55 HTTP/1.1" 502 150 2026-01-12 09:37:55,186 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a8427831b949bfeb55'): Retry(total=2, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:37:59,186 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a8427831b949bfeb55 2026-01-12 09:37:59,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a8427831b949bfeb55 HTTP/1.1" 502 150 2026-01-12 09:37:59,187 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a8427831b949bfeb55'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:38:07,187 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a8427831b949bfeb55 2026-01-12 09:38:07,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a8427831b949bfeb55 HTTP/1.1" 502 150 2026-01-12 09:38:07,189 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/696440a8427831b949bfeb55'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:45390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45404 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:38:23,189 - urllib3.connectionpool - DEBUG - Retry: /latest/node/696440a8427831b949bfeb55 2026-01-12 09:38:23,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/696440a8427831b949bfeb55 HTTP/1.1" 502 150 2026-01-12 09:38:23,190 - lava_callback - ERROR - Error processing callback for node 696440a8427831b949bfeb55: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696440a8427831b949bfeb55 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696440a8427831b949bfeb55 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/696440a8427831b949bfeb55 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50320 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:40:06,545 - lava_callback - INFO - Received callback for node 69621e6715f9f48f76d0e9ce INFO: 172.20.0.1:50332 - "POST /node/69621e6715f9f48f76d0e9ce HTTP/1.0" 202 Accepted 2026-01-12 09:40:06,552 - lava_callback - INFO - Starting async job submit for node 69621e6715f9f48f76d0e9ce 2026-01-12 09:40:06,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:40:06,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e6715f9f48f76d0e9ce HTTP/1.1" 502 150 2026-01-12 09:40:06,567 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e6715f9f48f76d0e9ce'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:40:06,567 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e6715f9f48f76d0e9ce 2026-01-12 09:40:06,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e6715f9f48f76d0e9ce HTTP/1.1" 502 150 2026-01-12 09:40:06,568 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e6715f9f48f76d0e9ce'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:40:08,568 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e6715f9f48f76d0e9ce 2026-01-12 09:40:08,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e6715f9f48f76d0e9ce HTTP/1.1" 502 150 2026-01-12 09:40:08,569 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e6715f9f48f76d0e9ce'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:40:12,570 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e6715f9f48f76d0e9ce 2026-01-12 09:40:12,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e6715f9f48f76d0e9ce HTTP/1.1" 502 150 2026-01-12 09:40:12,573 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e6715f9f48f76d0e9ce'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:42172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:40:20,573 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e6715f9f48f76d0e9ce 2026-01-12 09:40:20,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e6715f9f48f76d0e9ce HTTP/1.1" 502 150 2026-01-12 09:40:20,575 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e6715f9f48f76d0e9ce'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54384 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:40:36,575 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e6715f9f48f76d0e9ce 2026-01-12 09:40:36,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e6715f9f48f76d0e9ce HTTP/1.1" 502 150 2026-01-12 09:40:36,578 - lava_callback - ERROR - Error processing callback for node 69621e6715f9f48f76d0e9ce: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621e6715f9f48f76d0e9ce (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621e6715f9f48f76d0e9ce (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621e6715f9f48f76d0e9ce (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41768 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:41:06,590 - lava_callback - INFO - Received callback for node 69621e9e15f9f48f76d0e9d3 INFO: 172.20.0.1:41784 - "POST /node/69621e9e15f9f48f76d0e9d3 HTTP/1.0" 202 Accepted 2026-01-12 09:41:06,598 - lava_callback - INFO - Starting async job submit for node 69621e9e15f9f48f76d0e9d3 2026-01-12 09:41:06,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:41:06,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e9e15f9f48f76d0e9d3 HTTP/1.1" 502 150 2026-01-12 09:41:06,620 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e9e15f9f48f76d0e9d3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:41:06,620 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e9e15f9f48f76d0e9d3 2026-01-12 09:41:06,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e9e15f9f48f76d0e9d3 HTTP/1.1" 502 150 2026-01-12 09:41:06,621 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e9e15f9f48f76d0e9d3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:41:08,621 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e9e15f9f48f76d0e9d3 2026-01-12 09:41:08,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e9e15f9f48f76d0e9d3 HTTP/1.1" 502 150 2026-01-12 09:41:08,622 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e9e15f9f48f76d0e9d3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:41:12,623 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e9e15f9f48f76d0e9d3 2026-01-12 09:41:12,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e9e15f9f48f76d0e9d3 HTTP/1.1" 502 150 2026-01-12 09:41:12,624 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e9e15f9f48f76d0e9d3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:59126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59138 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:41:20,624 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e9e15f9f48f76d0e9d3 2026-01-12 09:41:20,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e9e15f9f48f76d0e9d3 HTTP/1.1" 502 150 2026-01-12 09:41:20,625 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621e9e15f9f48f76d0e9d3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41688 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:41:36,626 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621e9e15f9f48f76d0e9d3 2026-01-12 09:41:36,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621e9e15f9f48f76d0e9d3 HTTP/1.1" 502 150 2026-01-12 09:41:36,627 - lava_callback - ERROR - Error processing callback for node 69621e9e15f9f48f76d0e9d3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621e9e15f9f48f76d0e9d3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621e9e15f9f48f76d0e9d3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621e9e15f9f48f76d0e9d3 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:53140 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:41:46,644 - lava_callback - INFO - Received callback for node 69621ed315f9f48f76d0e9d8 INFO: 172.20.0.1:53156 - "POST /node/69621ed315f9f48f76d0e9d8 HTTP/1.0" 202 Accepted 2026-01-12 09:41:46,653 - lava_callback - INFO - Starting async job submit for node 69621ed315f9f48f76d0e9d8 2026-01-12 09:41:46,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:41:46,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621ed315f9f48f76d0e9d8 HTTP/1.1" 502 150 2026-01-12 09:41:46,683 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621ed315f9f48f76d0e9d8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:41:46,684 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621ed315f9f48f76d0e9d8 2026-01-12 09:41:46,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621ed315f9f48f76d0e9d8 HTTP/1.1" 502 150 2026-01-12 09:41:46,684 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621ed315f9f48f76d0e9d8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:41:48,685 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621ed315f9f48f76d0e9d8 2026-01-12 09:41:48,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621ed315f9f48f76d0e9d8 HTTP/1.1" 502 150 2026-01-12 09:41:48,686 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621ed315f9f48f76d0e9d8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:41:52,686 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621ed315f9f48f76d0e9d8 2026-01-12 09:41:52,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621ed315f9f48f76d0e9d8 HTTP/1.1" 502 150 2026-01-12 09:41:52,688 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621ed315f9f48f76d0e9d8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57292 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:42:00,688 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621ed315f9f48f76d0e9d8 2026-01-12 09:42:00,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621ed315f9f48f76d0e9d8 HTTP/1.1" 502 150 2026-01-12 09:42:00,689 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621ed315f9f48f76d0e9d8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54210 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:42:16,689 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621ed315f9f48f76d0e9d8 2026-01-12 09:42:16,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621ed315f9f48f76d0e9d8 HTTP/1.1" 502 150 2026-01-12 09:42:16,691 - lava_callback - ERROR - Error processing callback for node 69621ed315f9f48f76d0e9d8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621ed315f9f48f76d0e9d8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621ed315f9f48f76d0e9d8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621ed315f9f48f76d0e9d8 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43736 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:42:46,711 - lava_callback - INFO - Received callback for node 69621f0a15f9f48f76d0e9e3 INFO: 172.20.0.1:43752 - "POST /node/69621f0a15f9f48f76d0e9e3 HTTP/1.0" 202 Accepted 2026-01-12 09:42:46,719 - lava_callback - INFO - Starting async job submit for node 69621f0a15f9f48f76d0e9e3 2026-01-12 09:42:46,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:42:46,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f0a15f9f48f76d0e9e3 HTTP/1.1" 502 150 2026-01-12 09:42:46,742 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f0a15f9f48f76d0e9e3'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:42:46,742 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f0a15f9f48f76d0e9e3 2026-01-12 09:42:46,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f0a15f9f48f76d0e9e3 HTTP/1.1" 502 150 2026-01-12 09:42:46,743 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f0a15f9f48f76d0e9e3'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:42:48,743 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f0a15f9f48f76d0e9e3 2026-01-12 09:42:48,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f0a15f9f48f76d0e9e3 HTTP/1.1" 502 150 2026-01-12 09:42:48,745 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f0a15f9f48f76d0e9e3'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:42:52,745 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f0a15f9f48f76d0e9e3 2026-01-12 09:42:52,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f0a15f9f48f76d0e9e3 HTTP/1.1" 502 150 2026-01-12 09:42:52,746 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f0a15f9f48f76d0e9e3'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39740 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:43:00,746 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f0a15f9f48f76d0e9e3 2026-01-12 09:43:00,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f0a15f9f48f76d0e9e3 HTTP/1.1" 502 150 2026-01-12 09:43:00,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f0a15f9f48f76d0e9e3'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:43:16,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f0a15f9f48f76d0e9e3 2026-01-12 09:43:16,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f0a15f9f48f76d0e9e3 HTTP/1.1" 502 150 2026-01-12 09:43:16,754 - lava_callback - ERROR - Error processing callback for node 69621f0a15f9f48f76d0e9e3: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f0a15f9f48f76d0e9e3 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f0a15f9f48f76d0e9e3 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f0a15f9f48f76d0e9e3 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:43:46,731 - lava_callback - INFO - Received callback for node 69621f4015f9f48f76d0e9e8 INFO: 172.20.0.1:46306 - "POST /node/69621f4015f9f48f76d0e9e8 HTTP/1.0" 202 Accepted 2026-01-12 09:43:46,737 - lava_callback - INFO - Starting async job submit for node 69621f4015f9f48f76d0e9e8 2026-01-12 09:43:46,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:43:46,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f4015f9f48f76d0e9e8 HTTP/1.1" 502 150 2026-01-12 09:43:46,750 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f4015f9f48f76d0e9e8'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:43:46,750 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f4015f9f48f76d0e9e8 2026-01-12 09:43:46,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f4015f9f48f76d0e9e8 HTTP/1.1" 502 150 2026-01-12 09:43:46,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f4015f9f48f76d0e9e8'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:43:48,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f4015f9f48f76d0e9e8 2026-01-12 09:43:48,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f4015f9f48f76d0e9e8 HTTP/1.1" 502 150 2026-01-12 09:43:48,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f4015f9f48f76d0e9e8'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:43:52,752 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f4015f9f48f76d0e9e8 2026-01-12 09:43:52,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f4015f9f48f76d0e9e8 HTTP/1.1" 502 150 2026-01-12 09:43:52,753 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f4015f9f48f76d0e9e8'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37252 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:44:00,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f4015f9f48f76d0e9e8 2026-01-12 09:44:00,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f4015f9f48f76d0e9e8 HTTP/1.1" 502 150 2026-01-12 09:44:00,755 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f4015f9f48f76d0e9e8'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:53882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38224 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:44:16,755 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f4015f9f48f76d0e9e8 2026-01-12 09:44:16,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f4015f9f48f76d0e9e8 HTTP/1.1" 502 150 2026-01-12 09:44:16,757 - lava_callback - ERROR - Error processing callback for node 69621f4015f9f48f76d0e9e8: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f4015f9f48f76d0e9e8 (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f4015f9f48f76d0e9e8 (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f4015f9f48f76d0e9e8 (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:54218 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:44:26,809 - lava_callback - INFO - Received callback for node 69621f7615f9f48f76d0e9eb INFO: 172.20.0.1:54222 - "POST /node/69621f7615f9f48f76d0e9eb HTTP/1.0" 202 Accepted 2026-01-12 09:44:26,815 - lava_callback - INFO - Starting async job submit for node 69621f7615f9f48f76d0e9eb 2026-01-12 09:44:26,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:44:26,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f7615f9f48f76d0e9eb HTTP/1.1" 502 150 2026-01-12 09:44:26,853 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f7615f9f48f76d0e9eb'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:44:26,854 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f7615f9f48f76d0e9eb 2026-01-12 09:44:26,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f7615f9f48f76d0e9eb HTTP/1.1" 502 150 2026-01-12 09:44:26,854 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f7615f9f48f76d0e9eb'): Retry(total=3, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:44:28,855 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f7615f9f48f76d0e9eb 2026-01-12 09:44:28,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f7615f9f48f76d0e9eb HTTP/1.1" 502 150 2026-01-12 09:44:28,856 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f7615f9f48f76d0e9eb'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:44:32,856 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f7615f9f48f76d0e9eb 2026-01-12 09:44:32,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f7615f9f48f76d0e9eb HTTP/1.1" 502 150 2026-01-12 09:44:32,857 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f7615f9f48f76d0e9eb'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:42002 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:44:40,857 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f7615f9f48f76d0e9eb 2026-01-12 09:44:40,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f7615f9f48f76d0e9eb HTTP/1.1" 502 150 2026-01-12 09:44:40,859 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/69621f7615f9f48f76d0e9eb'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:48810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44006 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:44:56,859 - urllib3.connectionpool - DEBUG - Retry: /latest/node/69621f7615f9f48f76d0e9eb 2026-01-12 09:44:56,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69621f7615f9f48f76d0e9eb HTTP/1.1" 502 150 2026-01-12 09:44:56,860 - lava_callback - ERROR - Error processing callback for node 69621f7615f9f48f76d0e9eb: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f7615f9f48f76d0e9eb (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f7615f9f48f76d0e9eb (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/69621f7615f9f48f76d0e9eb (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:48120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44982 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:55:14,600 - lava_callback - INFO - Received callback for node 6964522e427831b949bfee5a INFO: 172.20.0.1:50954 - "POST /node/6964522e427831b949bfee5a HTTP/1.0" 202 Accepted 2026-01-12 09:55:14,616 - lava_callback - INFO - Starting async job submit for node 6964522e427831b949bfee5a 2026-01-12 09:55:14,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-01-12 09:55:14,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964522e427831b949bfee5a HTTP/1.1" 502 150 2026-01-12 09:55:14,748 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964522e427831b949bfee5a'): Retry(total=4, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:55:14,748 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964522e427831b949bfee5a 2026-01-12 09:55:14,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964522e427831b949bfee5a HTTP/1.1" 502 150 2026-01-12 09:55:14,749 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964522e427831b949bfee5a'): Retry(total=3, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:50970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50974 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:55:16,749 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964522e427831b949bfee5a 2026-01-12 09:55:16,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964522e427831b949bfee5a HTTP/1.1" 502 150 2026-01-12 09:55:16,751 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964522e427831b949bfee5a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) 2026-01-12 09:55:20,751 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964522e427831b949bfee5a 2026-01-12 09:55:20,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964522e427831b949bfee5a HTTP/1.1" 502 150 2026-01-12 09:55:20,752 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964522e427831b949bfee5a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:36202 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:55:28,753 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964522e427831b949bfee5a 2026-01-12 09:55:28,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964522e427831b949bfee5a HTTP/1.1" 502 150 2026-01-12 09:55:28,754 - urllib3.util.retry - DEBUG - Incremented Retry for (url='/latest/node/6964522e427831b949bfee5a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) INFO: 172.20.0.1:57508 - "GET / HTTP/1.0" 200 OK 2026-01-12 09:55:44,754 - urllib3.connectionpool - DEBUG - Retry: /latest/node/6964522e427831b949bfee5a 2026-01-12 09:55:44,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/6964522e427831b949bfee5a HTTP/1.1" 502 150 2026-01-12 09:55:44,756 - lava_callback - ERROR - Error processing callback for node 6964522e427831b949bfee5a: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6964522e427831b949bfee5a (Caused by ResponseError('too many 502 error responses')) urllib3.exceptions.ResponseError: too many 502 error responses The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 942, in urlopen return self.urlopen( ^^^^^^^^^^^^^ [Previous line repeated 2 more times] File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 932, in urlopen retries = retries.increment(method, url, response=response, _pool=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6964522e427831b949bfee5a (Caused by ResponseError('too many 502 error responses')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 284, in async_job_submit job_node = api_helper.api.node.get(node_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/latest.py", line 108, in get return self._get(f'node/{node_id}').json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/api/__init__.py", line 80, in _get resp = session.get( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 668, in send raise RetryError(e, request=request) requests.exceptions.RetryError: HTTPSConnectionPool(host='staging.kernelci.org', port=9000): Max retries exceeded with url: /latest/node/6964522e427831b949bfee5a (Caused by ResponseError('too many 502 error responses')) INFO: 172.20.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]